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BOARD 

January  13,  2009 

MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  INTELLIGENCE 

UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION, 
TECHNOLOGY  &  LOGISTICS 


SUBJECT:  Final  Report  of  the  Defense  Science  Board  (DSB)  Advisory  Group  on 
Defense  Intelligence  Task  Force  on  Operations  Research  Applications  for 
Intelligence,  Surveillance  and  Reconnaissance  (I  SR) 


I  am  pleased  to  forward  the  final  report  of  the  DSB  Advisory  Group  on  Defense 
Intelligence  Task  Force  on  Operations  Research  Applications  for  ISR  co-chaired  by  Dr. 
Robert  Lucky  and  Major  General  Richard  O’Lear,  USAF  (Ret).  This  Task  Force  was 
sponsored  by  the  Under  Secretary  of  Defense  for  Intelligence  (USD(I)). 

As  requested  in  the  Terms  of  Reference  (TOR)  the  Task  Force  was  asked  to  assess 
the  use  of  operations  research  (OR)  methods  within  the  DoD,  detennine  how  OR 
supports  DoD  decision  making  for  ISR  programs  and  recommend  needed  improvements. 

The  final  report  addresses  the  taskings  in  the  TOR  and  provides  two  test  cases  for 
using  OR  methodologies.  The  report  provides  recommendations  for  the  USD(I)  which 
include  establishing  an  organic  ISR  OR  oversight  function  within  USD(I)  as  well  as  a 
capability  to  conduct  OR  for  strategic  investment  decisions.  Further  recommendations 
address  the  details  of  the  OUSD(I)  OR  function. 


I  endorse  the  Task  Force’s  findings  and  recommendations  and  encourage  you  to 
review  the  report. 


WtefiAj* 


%  J 


Dr.  William  Schneider,  Jr. 
DSB  Chairman 
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December  9,  2008 


MEMORANDUM  FOR  THE  CHAIRMAN,  DEFENSE  SCIENCE  BOARD 

SUBJECT:  Final  Report  of  the  Defense  Science  Board  (DSB)  Advisory  Group  on  Defense 

Intelligence  Task  Force  on  Operations  Research  Applications  for  Intelligence,  Surveillance  and 
Reconnaissance  (ISR) 

Attached  is  the  final  report  of  the  DSB  Advisory  Group  on  Intelligence  Task  Force  on 
Operations  Research  Applications  for  ISR.  The  Task  Force  was  asked  to  examine  the  use  of 
operations  research  (OR)  methods  to  support  ISR  decision  making  within  the  Department  of 
Defense  (DoD). 

The  Terms  of  Reference  asked  the  Task  Force  to  survey  the  departments  and  agencies  to 
determine  how  much  OR  is  being  performed;  assess  how  the  results  of  OR  are  being  used  in 
decision  making;  recommend  a  test  case(s)  for  using  OR  methodologies;  and  recommend  steps 
the  Department  can  take  to  institutionalize  the  use  of  OR  in  future  DoD  decision  making. 

The  Task  Force  reviewed  the  application  of  OR  -  which  it  views  as  decision  support  analysis 
based  on  information  collected  and  analyzed  using  the  scientific  method  -  throughout  the  DoD 
and  the  Services  focusing  primarily  on  the  resources  available  to  conduct  OR,  as  opposed  to 
reviewing  specific  programs.  The  Task  Force  examined  the  commitment  of  decision  makers  to 
the  use  of  OR  and  the  extent  to  which  these  resources  reflected  the  application  of  standard  OR 
processes  and  practices  and  the  manner  in  which  the  results  appeared  to  be  employed.  The  Task 
Force  also  looked  at  several  private-sector  OR  application  models  and  at  recent  and  historical 
uses  of  OR  in  support  of  national  security  requirements. 

Based  on  its  inquiry  the  Task  Force  arrived  at  the  following  observations: 

•  Operations  Research  represents  a  powerful  tool  to  help  improve  the  quality  of 
investment  decision  making  by  illuminating  key  issues,  assumptions,  and  sources  of 
information. 

•  Operations  Research  is  applied  inconsistently  throughout  the  Defense  and  ISR 
communities  and  each  lacks  standard  OR  processes  and  practices,  and  consistent 
organizational  models  or  commitments. 


•  OR  and  the  use  of  it  can  be  strengthened  in  the  Defense  and  ISR  communities 
through  effective  institutionalization. 

•  The  utility  of  OR  can  be  more  firmly  established  through  appropriate  test  cases. 

Based  on  its  findings,  the  Task  Force  recommends  establishing  an  organic  OR  oversight  function 
and  investment  decision  capability  within  the  USD(I).  In  response  to  the  TOR,  the  Task  Force 
recommends  two  test  cases,  one  related  to  the  balance  in  the  intelligence  cycle  and  the  other 
related  to  investment  decision  making  in  biometrics  technologies. 

The  attached  report  provides  the  rationale  for  the  Task  Force’s  findings  and  recommendations, 
reflects  on  the  historical  use  of  OR,  details  current  OR  practices,  specifies  attributes  required  for 
the  test  cases,  and  offers  potential  institutionalization  models. 

We  appreciate  the  contributions  made  by  the  Department  of  Defense,  Intelligence  Community, 
and  private  sector  professionals  who  took  the  time  to  provide  us  with  their  knowledge  and 
expertise;  the  members  of  this  study;  the  Executive  Secretary;  and  the  DSB  Secretariat  and  its 
military  assistant. 


Co-Chairman 


Co-Chairman 
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Executive  Summary 

The  Advisory  Group  on  Defense  Intelligence  (the  Task  Force)  of  the  Defense  Science 
Board  (DSB)  was  tasked  by  the  Under  Secretary  of  Defense  for  Intelligence  (USD(I))  to 
examine  the  manner  and  extent  to  which  Operations  Research  (OR)  is  employed  by  the 
Department  of  Defense;  how  OR  can  be  used  to  support  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  decision  making;  and  the  manner  in  which  OR  can  be 
institutionalized  in  the  Department  of  Defense  (DoD). 

The  Task  Force  reviewed  the  application  of  OR1  -  which  it  views  as  decision  support 
analysis  based  on  information  collected  and  analyzed  using  the  scientific  method  - 
throughout  the  DoD  and  the  Services.  The  review  was  done  on  a  selective  basis, 
focusing  principally  on  the  resources  available  to  conduct  OR  as  opposed  to  specific 
programs.  The  Task  Force  looked  at  decision  makers'  commitment  to  the  use  of  OR,  the 
extent  to  which  resources  reflected  the  application  of  standard  processes  and  practices, 
and  the  manner  in  which  the  results  of  resources  appeared  to  be  employed  by  the 
organizations  of  which  they  are  a  part.  The  Task  Force  also  looked  at  several  private- 
sector  OR  application  models  (notably  FedEx),  and  at  recent  and  historical  use  of  OR  in 
support  of  national  security  requirements. 

Four  aspects  of  our  inquiry  stand  out: 


1.  Operations  Research  represents  a  powerful  tool  to  help  improve  the  quality 
of  investment  decision  making  by  illuminating  key  issues,  assumptions,  and 
sources  of  information. 

2.  Operations  Research  is  applied  inconsistently  throughout  the  Defense  and 
ISR  communities.  These  communities  do  not  possess  standard  OR  processes 
and  practices,  a  consistent  organizational  model,  or  a  consistent 
commitment  to  the  use  of  OR. 

3.  OR  -  and  its  use  -  can  be  strengthened  in  the  Defense  and  ISR  communities 
through  effective  institutionalization.  The  Task  Force  commends  to  the 
USD(I)  for  further  consideration  models  employed  by  the  private  sector  (of 
which  FedEx  appears  to  be  a  strong  examples)  and  the  Army  (TRAC  and  CAA). 
The  Military  Operations  Research  Society  (MORS)  is  a  domain  expert 
resource  that  should  be  used  in  building  a  plan  for  institutionalization. 

4.  The  utility  of  OR  can  be  more  firmly  established  through  appropriate  test 
cases.  The  Task  Force  points  to  two  cases  for  USD(I)  consideration: 
Biometrics  and  Investment  in  a  Balanced  Intelligence  Cycle,  with  specific 
emphasis  on  Unmanned  Aerial  Systems  (UASs). 


1  The  analytical  study  of  military  problems  undertaken  to  provide  responsible  commanders  and  staff  agencies  with  a  scientific  basis  for 
decision  on  action  to  improve  military  operations.  Also  called  operational  research;  operations  analysis."  (JCS  Pub  1-02) 
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The  Task  Force  believes  Operations  Research  can  strengthen  the  quality  of  ISR  decision 
making  in  terms  of  the  ability  of  investment  decisions  to  more  clearly  reflect 
requirements,  alternatives,  resources,  and  technologies.  However,  OR  is  currently 
employed  throughout  DoD  and  the  Services  on  an  inconsistent  basis,  reflecting  varying 
levels  of  leadership  commitment  and  support,  a  lack  of  standards  regarding  OR  training 
and  methodologies,  and  uncertain  career  recognition  and  prospects  for  OR 
practitioners.  Although  the  Federal  Acquisition  Regulations  (FAR)  require  the  use  of  OR 
in  support  of  Major  Acquisition  Programs,  OR  is  used  inconsistently  in  ISR  investment 
decision  making,  even  on  major  systems,  and  the  resources  employed  do  not  reflect 
standard  practice.  In  general,  OR  is  used  to  configure  existing  resources  or  to  support 
(or  refute  the  need  for)  additional  "buys"  of  existing  systems.  OR  is  also  used  to  support 
complex  mission  management  decisions  regarding  operational  systems.  Overall, 
however,  the  use  of  Operations  Research  is  diminished  by  a  seeming  lack  of  leadership 
support  for  its  application  in  support  of  program  investment,  career  investment,  and 
sufficient  resource  investment  to  create  sustained  excellence  in  its  practice  throughout 
the  ISR  and  larger  DoD  communities.  In  contrast,  OR  (and  science  method-based 
analysis)  were  disciplines  prized  for  their  ability  to  help  decision  makers  meet  significant 
challenges,  particularly  in  support  of  U.S.  military  operations  during  World  War  II,  in 
determining  force  mix  and  sufficiency  after  WWII,  and  in  rationally  mobilizing  the  U.S. 
industrial  base  in  support  of  national  security  requirements.  While  there  exist  islands  of 
OR  practice  today  -  some  of  them  quite  competent  -  in  DoD  and  the  Services, 
institutional  capacity  to  apply  science-based  analysis  to  complex  decisions  is  far  less 
than  it  has  been  in  the  past,  and  less  than  it  should  be. 

Institutionalization  of  OR  for  ISR  investment  decision  making  requires  a  sound 
management  model  and  the  sustained  support  of  DoD  leaders,  both  in  its  application 
and  sustainment  of  the  resources  necessary  to  keep  OR  a  robust  discipline  throughout 
the  Department  and  the  Services.  Such  commitment  will  be  required  to  reach  a  level  of 
competent  OR  practice  throughout  the  ISR  community.  At  present,  competent  OR  cells 
exist,  but  are  poorly  connected  as  a  professional  community  and  inconsistently  utilized 
by  decision  makers.  The  Task  Force  believes  OR  should  be  institutionalized,  practiced 
consistently  (from  a  standards  and  methodology  perspective),  and  used  routinely  to 
support  high  value  decisions  involving  complex  requirements  and  significant  resources. 
The  Task  Force  understands  clearly  that  OR  does  not  "make"  decisions;  rather  OR  can 
and  should  make  visible  the  sources  of  data  and  information  used  to  make  decisions, 
help  drive  the  development  of  consistent  assumptions  and  requirements,  and  give  an 
objective  basis  against  which  decisions  can  be  weighed.  The  Task  Force  recognizes  that 
complex  decisions  exist  within  a  wider  context  of  policy  and  changing  requirements;  OR 
provides  an  objective  "anchor"  that  decision  makers  should  consider. 

The  Task  Force  recommends  that  the  Federal  Express  (FedEx)  management  model  be 
given  strong  USD(I)  consideration  as  a  sound  example  of  the  use  of  OR  in  the  private 
sector.  In  addition,  the  work  being  done  by  the  Army's  Training  and  Doctrine 
Command's  (TRADOC)  Analyses  Center  (TRAC)  and  the  Center  for  Army  Analysis  (CAA) 
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provide  good  examples  of  utilizing  OR  within  the  government.  The  private  sector  model 
used  by  FedEx  employs  OR  at  a  corporate  level  to  decide  which  lines  of  business  are 
competitive  and  at  what  level;  employs  OR  at  the  line-of-business  level  for  investment 
decision  making  and  configuration  of  operational  resources;  and  provides  corporate 
oversight  to  ensure  the  presence  of  consistent  OR  standards  methodology  throughout 
the  company's  family  of  businesses.  FedEx  does  not  "require"  the  use  of  OR  instead  the 
FedEx  leadership  commitment  to  OR-based  decision  making  creates  a  culture  which 
gives  its  practitioners  substantial  influence  during  high  value  decision  making.  At  the 
same  time,  the  Task  Force  believes  that  TRAC  may  represent  a  strong  example  of  an 
embedded,  "line-of-business"  capability,  analogous  to  the  OR  capabilities  in  FedEx's 
various  lines-of-business.  Overall,  the  Task  Force  recommends  the  USD(I)  establish  an 
organic2  ISR  OR  oversight  function  that  requires,  sustains,  and  reviews  OR  standard 
processes  and  practices  in  DoD  ISR  investment  decisions. 

In  response  to  the  TOR,  the  Task  Force  recommends  two  ISR  test  cases.  The  first  related 
to  investment  decision  making  in  biometrics  technologies  in  support  of  ISR 
requirements;  and  the  second  to  investment  decision  making  for  balance  in  the  overall 
intelligence  cycle,  and  specifically  in  regard  to  UASs.  These  test  cases  should  be 
structured  to  illuminate  challenges  and  opportunities  to  strengthen  OR  decision  making 
in  the  ISR  investment  domain.  The  biometrics  test  case  provides  an  opportunity  to  help 
biometrics  meet  its  ISR  potential  through  stronger  investment  strategy  decision  making, 
creation  of  a  real  biometrics  information  architecture,  and  development  of  a  biometrics 
information  sharing  environment  useful  to  both  intelligence  analysts  and  warfighters. 

Building  a  more  effective  OR  capability  to  support  ISR  is  a  difficult  but  meaningful 
challenge.  The  Task  Force  believes  that  the  Military  Operations  Research  Society 
(MORS)  is  a  resource  the  USD(I)  can  use  to  help  implement  this  study's 
recommendations. 

In  summary,  the  decision  maker  must  commit  to  using  the  OR  results  in  support  of 
decisions  to  be  made.  While  it  is  not  realistic  to  believe  that  an  important  ISR 
investment  decision  will  be  driven  solely  by  OR  analytical  results,  it  should  be  clear  to 
stake-holders  that  the  results  were  carefully  considered  in  the  decision-making  process. 

The  Task  Force  looks  forward  to  discussing  with  the  USD(I)  and  the  ISR  community  its 
recommendations  and  is  grateful  for  this  opportunity  to  serve  the  larger  national 
security  community. 


2  An  in-house  OR  function. 
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Chapter  1:  Introduction  and  Tasking 

As  a  result  of  the  recent  High  Value  Targets  (HVT)  Study  performed  by  Edge  Consulting, 
Inc.  to  examine  the  application  of  ISR  assets  in  support  of  operations  in  the  U.S.  Central 
Command's  Area  of  Responsibility  (USCENTCOM  AOR),  senior  leaders  in  the  Office  of 
the  Under  Secretary  of  Defense  for  Intelligence  (OUSD(I)),  Program  Analysis  and 
Evaluation  (PA&E),  and  the  Joint  Staff  engaged  in  a  discussion  about  the  merits  of 
operations  research  in  support  of  planning,  programming,  and  operations.3  This 
discussion  led  to  a  request  by  the  USD(I)  to  the  Defense  Science  Board  (DSB)  to  perform 
a  study  on  the  use  of  Operations  Research  methods  to  support  decision  making  within 
the  Department,  with  a  focus  on  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 
investment  choices.  The  DSB  assigned  this  task  to  its  Advisory  Group  on  Defense 
Intelligence  (the  Task  Force).  The  Terms  of  Reference  (TOR)  directed  the  Task  Force  to 
examine  the  manner  and  extent  to  which  OR  is  employed  by  the  Department  of  Defense 
(DoD),  how  OR  can  be  used  to  support  ISR  decision  making,  and  the  manner  in  which  OR 
can  be  institutionalized  in  the  DoD.  The  TOR  included  the  following  four  tasks: 

1.  Survey  the  departments  and  agencies  to  determine  how  much  OR  is  being 
performed; 

2.  Assess  how  the  results  of  that  OR  are  being  used  in  decision  making; 

3.  Based  on  these  findings,  recommend  a  test  case  for  using  OR  methodologies 
to  improve  DoD  ISR  investment  decisions;  and 

4.  Recommend  steps  the  Department  can  take  to  "institutionalize"  the  use  of 
OR  in  DoD  decision  making  in  the  future. 

The  Task  Force  discussed  the  scope  of  the  OR  inquiry  that  would  best  serve  the  USD(l)'s 
needs  and  considered  limiting  the  inquiry  to  an  examination  of  OR  as  a  component  of 
technology  and  acquisition  strategy.  The  Task  Force  also  considered  extending  the 
inquiry  to  ways  in  which  OR  can  be  used  to  strengthen  operational  planning.  In  the  end, 
the  Task  Force  concluded  that  a  broad  inquiry  was  appropriate  to  the  TOR  tasks 
provided  by  the  USD(I).  Such  an  inquiry  included  examination  and  discussion  of  OR  as  a 
component  of  operational  planning,  as  an  element  in  resource  allocation  in  support  of 
operations,  and  as  an  element  of  acquisition  and  technology  strategy  pertinent  to 
supporting  resource  allocation  and  operational  planning.  The  Task  Force  believes  this 
broad-based  approach  is  particularly  relevant  to  the  ISR  challenges  facing  the 
Department  now  and  in  the  foreseeable  future.  Secretary  of  Defense  Robert  Gates 


3  The  Office  of  the  Undersecretary  of  Defense  (Intelligence),  Program  Analysis  and  Evaluation,  and  the  Joint  Staff  co-sponsored  Edge 
Consulting,  Inc.  to  deliver  a  series  of  quantitative  ISR  performance  analyses.  The  Department's  objective  was  to  enrich  the 
standard  methods  -  interviews,  surveys,  war  games  -  with  quantitative  analysis  of  empirical  operations  and  intelligence  performance  data. 
As  one  example,  the  Department  tasked  Edge  with  quantifying  the  contribution  of  a  dozen  Intelligence  Surveillance  and  Reconnaissance 
(ISR)  systems  to  the  special  operations  campaign  against  High  Value  Targets  (HVT).  Working  in  close  cooperation  with  the  Department, 
U.S.  Central  Command,  and  Special  Operations  Forces,  Edge  gathered  over  one  million  empirical  records  associated  with  thousands  of 
HVT  operations.  Operations  Research  techniques  were  adapted  and  applied  to  quantify  the  contribution  of  ISR  systems  to  successful 
operations.  Causal  factor  analysis  was  applied  to  understand  root  causes  of  poor  performance.  The  Department  and  special  operations 
forces  applied  these  results  to  adjust  both  acquisition  and  operational  decisions. 
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described  some  of  these  challenges  -  related  to  requirements  of  ever  increasing 
complexity  and  constrained  resources  -  in  a  speech  on  September  29,  2008  at  the 
National  Defense  University.4  His  speech  highlighted  the  challenges  associated  with 
employing  complex  platforms  and  capabilities  against  an  ever-changing  set  of  missions 
and  requirements.  These  challenges  apply  clearly  to  current  and  prospective  ISR 
investments. 

Given  this  complex  environment,  the  Task  Force  believes  that  OR  can  strengthen  end- 
to-end  analysis  associated  with  coupling  long-term  investment  strategies,  requirements, 
operational  concepts,  and  operational  plans.  The  Task  Force  received  22  briefings  and 
questioned  representatives  of  the  Services,  various  operational  agencies,  planning  and 
evaluation  specialists  from  PA&E,  and  program  management  cadres  within  the  DoD  and 
Office  of  the  Director  of  National  Intelligence  (ODNI).  The  Task  Force  also  spoke  with 
industry  representatives  who  provided  useful  insights  regarding  ways  in  which  OR  can 
be  used  for  strategic  investment  planning  and  in  support  of  missions  associated  with 
specific  lines  of  business.  Appendix  C  lists  the  organizations  that  briefed  Task  Force. 

The  Task  Force  believes  its  broad  approach  to  the  TOR  tasking  provides  the  USD(I)  with 
insights  pertinent  to  a  full  range  of  ISR  investment  needs,  ranging  from  long-range 
technology  strategy  to  the  efficient  use  of  existing  ISR  resources.  In  regard  to 
methodology,  the  Task  Force  focused  principally  on  the  resources  available  to  conduct 
OR,  instead  of  reviewing  specific  programs,  a  task  beyond  the  means  and  time  available 
for  this  effort.  The  Task  Force  looked  at  the  extent  to  which  these  resources  reflected 
the  application  of  standards  and  the  manner  in  which  the  results  appeared  to  be 
employed  by  the  respective  organizations.  This  approach  allowed  the  Task  Force  to 
infer  a  lack  of  OR  use  and  standard  processes  and  practices  in  ISR  investment  decision 
making.  Inquiries  into  specific  programs  and  investments  could  more  precisely  establish 
the  manner  and  competence  with  which  OR  has  been  or  is  being  employed. 


4  As  delivered  by  Secretary  of  Defense  Robert  M.  Gates,  Washington,  D.C.,  Monday,  September  29,  2008.  See: 
http://www.defenselink.  mil/speeches/speech. aspx?speechid=1279 
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Chapter  2:  The  Task  Force's  Understanding  of  Operations  Research 

Joint  Chiefs  of  Staff  Publication  1-02  (JCS  Pub  1-02),  Department  of  Defense  Dictionary  of 
Military  and  Associated  Terms,  defines  Operations  Research  as  "The  analytical  study  of 
military  problems  undertaken  to  provide  responsible  commanders  and  staff  agencies 
with  a  scientific  basis  for  decision  on  action  to  improve  military  operations."  Operations 
Research  may  also  be  referred  to  as  Operational  Research  or  Operations  Analysis. 

The  Task  Force  received  numerous  briefings  from  Operations  Research  (OR) 

practitioners  from  a  variety  of  organizations.  Information 
gleaned  from  these  briefings  convinced  the  Task  Force  that 
the  application  of  structured  decision  making  tools  and 
processes  can  provide  significant  assistance  to  the  decision 
maker  in  the  analysis  of  complex  operational  and  investment 
decisions.  OR  can  increase  significantly  the  decision  maker's 
insight  and  also  improve  the  quality  of  the  debate  around 
the  key  decision  issues. 

As  noted  in  Chapter  1,  the  Task  Force's  understanding  of  OR 
-  decision  support  analysis  based  on  information  collected  and  analyzed  using  the 
scientific  method  -  broadly  encompasses  analysis  of  operations  as  well  as  strategic 
technology,  investment,  and  acquisition  planning  in  support  of  operations.  OR 
practitioners  have  a  variety  of  tools  at  their  disposal.  These  tools  consist  of  applied 
mathematical  techniques  such  as  statistical  and  data  analysis,  linear  and  non-linear 
regression,  queuing  theory,  stochastics,  as  well  as  linear  and  non-linear  programming. 
Additionally,  practitioners  have  access  to  a  variety  of  off-the-shelf  models  and 
simulations,  as  well  as  commercial  software  packages.  For  those  problems  where  off- 
the-shelf  tools  are  not  practicable,  OR  practitioners  can  develop  models  and  simulations 
using  the  techniques  listed  above. 

It  is  the  Task  Force's  opinion  that  OR  should  be  used  as  an  important  component  of  an 
analytical  approach  to  decision  making.  Information  provided  to  the  Task  Force  by 
briefers  also  assisted  in  formulating  a  picture  of  what  good  OR  practices  should  entail. 

2.1  Desired  Attributes  of  Operations  Research 

The  Task  Force  viewed  OR  as  a  scientific  method;  a  method  based  on  gathering 
empirical  and  measurable  evidence  or  data  and  subjecting  it  to  repeatable  principles  of 
reasoning.  These  principles  of  reasoning  often  involve  the  formulation  and  testing  of 
hypotheses.  This  process  requires  a  thorough  identification  of  the  problem  to  be 
solved,  which  includes  advance  agreement  by  decision  makers  on  the  objective 
function,5  metrics,  data  provenance,  and  the  study  assumptions  and  boundary 


OR  CAN  SIGNIFICANTLY 
INCREASE  THE  DECISION 
MAKER'S  INSIGHT  AND 
IMPROVE  THE  QUALITY  OF 
DEBATE  AROUND 
KEY  DECISION  ISSUES. 


5  The  Task  Force  considered  at  length  the  manner  in  which  valid  objective  functions  could  be  employed  in  the  OR  analysis  of  ISR 
investments.  We  considered  established  "attrition  models"  by  which  the  effectiveness  of  a  system  is  assessed  as  a  function  of  the  extent 
to  which  it  contributes  to  the  attrition  of  adversary  forces,  as  well  as  the  extent  to  which  an  ISR  system  can  reduce  the  level  of  uncertainty 
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conditions  -  all  of  which  are  keys  to  solid  analysis.  The  best  use  of  OR  occurs  when 
employed  by  a  team  composed  of  operators  and  analysts  capable  of  investigating  the 
entire  problem  and  conducting  analyses  without  bias.  In  so  doing,  a  good  OR  team  will 
seek  agreement  on  assumptions  and  objective  functions,  and  advocacy  from  all  key 
stakeholders  up-front,  and  continue  to  cultivate  that  advocacy  throughout  the  analytic 
process. 

2.2  Advantages  of  Operations  Research 

Good  analysis  involves  all  decision  makers  and  stakeholders  from  the  very  beginning. 
This  involvement  includes,  but  is  not  limited  to,  agreement  on  the  actual  questions  to  be 
answered  by  the  analysis  as  well  as  all  assumptions  and  constraints  pertinent  to  the 
decision  to  be  made.  Such  involvement  by  senior  leadership  facilitates  periodic  reviews 
and  assessments  of  progress,  and  provides  the  opportunity  to  redirect  or  refine 
guidance  to  the  study  team  if  needed.  A  repeatable  process  based  on  the  scientific 
method  enhances  the  likelihood  that  the  analysis  will  be  objective  and  the  results  will 
provide  the  organizational  leadership  with  the  analytic  underpinnings  to  make  sound, 
informed  decisions.  Structured  analyses  will  provide  decision  makers  with  clear 
traceability  from  study  requirements  to  final  results.  In  addition,  analyses  can  often 
illuminate  problems  previously  unidentified  or  unknown  to  senior  leadership. 

2.3  ISR  Decisions  to  which  Operations  Research  is  Applicable 

OR  readily  lends  itself  to  assisting  organizational  leadership  in  making  high-value 
decisions  in  which  significant  consequences  and  resources  are  at  stake.  Such  decisions 
involve  risk  and  uncertainty,  and  are  often  driven  by  complex  requirements  involving 
multiple  stakeholders  with  competing  objectives.  These  analytic  processes  can  be  used 
across  all  ISR  disciplines  focused  on  a  prioritized  list  of  current  and  future  threats  and 
missions.  Tailored  approaches  can  be  applied  to  the  entire  intelligence  cycle  including 
system  requirements  analysis,  asset  tasking,  data  collection,  processing  and 
exploitation,  as  well  as  intelligence  analysis. 

2.4  Conditions  under  which  Operations  Research  can  be  Institutionalized 

OR  cannot  be  institutionalized  without  strong  commitment,  support,  and  advocacy  from 
senior  leadership,  which  is  assumed  to  have  the  authority  to  implement  decisions 
influenced  by  the  OR  process.  The  senior  leadership  to  which  the  Task  Force  is  referring 
includes  those  individuals  responsible  for  the  outcome  of  the  decision  process,  that  is, 
those  that  will  make  the  decision  based  upon  various  inputs. 

This  approach  requires  senior  leadership  to  assemble  and  sustain  a  critical  mass  of  OR 
resources  to  include  a  well-trained  and  adequately-funded  staff.  OR  professionals  need 
career  development  and  prospects.  The  Task  Force  found  noteworthy  the  approach 


regarding  a  question  of  intelligence  interest.  The  Task  Force  recognizes  that  establishing  objective  functions  will  continue  to  challenge 
decision  makers,  although  we  believe  good  work  has  been  done  to  establish  Key  Performance  Parameters  (KPPs)  for  many  ISR  systems. 
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employed  by  the  Army  (TRAC)  in  building  and  sustaining  a  cadre  of  competent  OR 
professionals. 

The  essential  requirement  for  effective  OR  in  support  of  decision  making  is  the  support, 
confidence,  and  early  participation  of  the  decision  maker.  The  decision  maker  must 
ensure  that  the  question  asked,  the  assumptions  made,  the  objective  function  defined, 
and  the  metrics  chosen  all  meet  the  needs  of  the  decision  process. 

A  decision  maker  who  demands  good  analytical  support  for  decisions  will  get  it. 
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Chapter  3:  Operations  Research  -  Historical  Context 

The  Task  Force  examined,  at  a  high  level,  OR  from  a  historical  perspective.  The  Task 
Force  looked  at  OR  in  the  context  of  its  development  and  use  principally  in  the  U.S.,  with 
an  understanding  that  the  OR  discipline  is  not  confined  to  the  U.S.  This  review  of  OR's 
historical  context  (from  World  War  I  to  present  day  requirements  and  acquisition 
processes)  provided  the  Task  Force  with  useful  insights  regarding  the  need  for 
standards,  involvement  of  operators  alongside  analysts,  and  senior  leadership 
commitment. 

World  War  I 

During  World  War  I  civilian  scientists  on  both  sides  of  the  Atlantic  joined  together  to 
apply  scientific  methods  to  defeating  the  forces  of  the  Triple  Alliance,  specifically  the 
submarine  threat.  In  1915,  after  reading  one  of  Thomas  Edison's  speeches  on  military 
preparedness  published  in  The  New  York  Times,  Secretary  of  the  Navy  Josephus  Daniels 
came  upon  the  idea  of  the  Naval  Consulting  Board.  After  discussions  with  Edison,  it  was 
determined  that  a  board  consisting  of  the  brightest  scientific  and  industrial  minds  in  the 
country  would  be  convened  under  Edison's  chairmanship.  Its  primary  purpose  was  to 
advise  the  Navy  on  systems  acquisition.  Edison,  fully  aware  of  the  threat  the  U-boat 
posed  to  merchant  shipping  and  Britain's  survival,  focused  his  talents  on  developing 
systems  and  tactics  to  be  used  in  the  fledgling  anti-submarine  warfare  (ASW)  effort.  Fie 
employed  many  applied  mathematics  techniques,  such  as  time-motion  analysis  to 
understand  the  effects  of  zigzagging  in  disrupting  fire  control  solutions;  graphical  and 
statistical  data  analysis  to  understand  U-boat  operations;  threat  system  assessments; 
and  military  wargaming.6  That  same  year  the  British  Admiralty  instituted  the  Board  of 
Invention  and  Research  to  encourage  the  naval  service  to  solicit  expert  technical 
assistance  in  solving  tactical  problems.  Chaired  by  Sir  Jackie  Fisher,  former  First  Sea 
Lord,  it  was  comprised  of  numerous  scientists  and  divided  into  six  science  and 
technology  sections.  Of  these.  Section  II:  Submarines  and  Wireless  Telegraphy,  was  the 
most  important.  In  the  two  years  of  the  Board's  existence,  its  members  evaluated  over 
41,000  submitted  inventions.7 

British  use  of  OR  during  World  War  II 

Operations  Research  (Operational  Research  in  the  United  Kingdom  and  Canada)  was 
formally  established  in  the  late  1930's  at  a  time  when  Adolf  Hitler  announced  his 
intentions  to  create  a  Luftwaffe  equal  to  the  combined  air  forces  of  Britain  and  France. 

To  defend  Britain  against  an  attack  from  Germany,  the  British  government  established 
radar  installations  along  the  coast  to  provide  the  Royal  Air  Force  (RAF)  with  early 
warning  of  incoming  German  strikes.  However,  in  1938,  tactical  exercises  demonstrated 
that  the  system  fell  short  of  its  operational  requirements.  A.P.  Rowe,  Superintendent  of 


6  Scott,  Lloyd  N.,  Naval  Consulting  Board  of  the  United  States,  Government  Printing  Office,  Washington,  D.C.,  1920. 

7  Hackmann,  Willem,  Seek  &  Strike:  Sonar,  Anti-Submarine  Warfare  and  the  Royal  Navy  1914-54,  Her  Majesty's  Stationary  Office  (HMSO), 
London,  1984. 
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the  Bawdsey  Research  Station,  proposed  that  research  into  operational  aspects  rather 
than  technical  aspects  of  the  system  should  be  conducted  -  thus  coining  the  phrase 
"operational  research."8, 9  These  studies  led  to  the  establishment  of  formal  techniques 
of  command  and  control  of  fighter  versus  bomber  operations  in  the  Battle  of  Britain. 

Operational  Research  spread  to  other  organizations  throughout  the  British  military 
establishment.  Operational  Research  sections  in  the  Coastal  Command  and  the 
Admiralty  were  responsible  for  developing  convoy  procedures  to  keep  Britain's  lifeline 
to  North  America  open,  perfecting  the  use  of  airborne  radar  in  detecting  U-boats,  and 
shutting  down  the  U-boat  transit  lanes  from  bases  along  the  Bay  of  Biscay  to  open 
ocean  operating  areas.10,  n'  12  Other  sections  were  established  to  support  the  Army 
(primarily  the  Antiaircraft  Command),  as  well  as  RAF  Bomber  Command  where  studies 
in  navigation  accuracy,  radar  performance,  and  the  employment  of  chaff  contributed  to 
improved  bombing  accuracy  and  aircraft  survivability  during  the  area  bombing  of 
German  cities.13  Recent  studies  in  the  United  Kingdom  indicate  that  the  area  bombing 
campaign  undertaken  by  Bomber  Command  in  1942,  and  espoused  by  the  Commander- 
in-Chief,  Sir  Arthur  Harris,  may  have  been  validated  by  the  misapplication  of  operations 
research  and  was  of  less  impact  on  the  German  war  economy  and  civilian  morale  then 
was  believed  at  the  time.14 

US.  use  of  OR  during  World  War  II 

American  scientists  became  involved  in  the  war  effort  in  June  1940  with  the  formation 
of  the  National  Defense  Research  Committee  (NDRC)  and  the  subsequent  formation  of 
the  Office  of  Scientific  Research  and  Development  (OSRD)  under  Dr.  Vannevar  Bush.15, 
16  Like  the  Naval  Consulting  Board  of  World  War  I,  both  the  NDRC  and  OSRD  were 
comprised  of  some  of  the  brightest  minds  from  academia  and  industry.  Upon  the  entry 
of  the  United  States  into  the  war,  the  Navy  took  the  lead  in  establishing  operations 
research  organizations.  In  March  1942  the  Mine  Warfare  Operations  Research  Group 
(MWORG)  was  established  at  the  Naval  Ordnance  Laboratory.  MWORG  went  through 
numerous  reorganizations  throughout  the  war,  and  ultimately  supported  the  Army  Air 
Forces  XXI  Bomber  Command  in  planning  Operation  Starvation  in  the  Far  East  -  the 


Cunningham,  W.  Peyton,  et  al,  "Of  Radar  and  Operations  Research:  An  Appreciation  of  A.P.  Rowe  (1898-1976),"  Operations  Research, 
Volume  32,  Number  4  (July-August  1984),  pp.  958-967. 

9  Larnder,  Harold,  "The  Origin  of  Operational  Research,"  Operations  Research,  Volume  32,  Number  2  (March  -  April  1984),  pp.  465-475. 

10  Falconer,  N.,  "On  the  Size  of  Convoys:  An  Example  of  the  Methodology  of  Leading  Wartime  OR  Scientists,"  Operational  Research 
Quarterly  (1970-1977),  Volume  27,  Number  2,  Part  1  (1976),  pp.  315-327. 

11  McCue,  Brian,  U-boats  in  the  Bay  of  Biscay:  An  Essay  in  Operations  Analysis,  National  Defense  University  Press,  Washington,  DC,  1990. 

12  McCloskey,  Joseph  F.,  "British  Operational  Research  in  World  War  II,"  Operations  Research,  Volume  35,  Number  3  (May  -  June  1987), 
pp.  453-470. 

13  Ibid. 

14  Kirby,  M.  and  R.  Capey,  "The  Area  Bombing  of  Germany  in  World  War  II:  An  Operational  Research  Perspective,"  The  Journal  of  the 
Operational  Research  Society,  Volume  48,  Number  7  (July  1997),  pp.  661-677. 

15  Meigs,  Montgomery  C.,  Slide  Rules  and  Submarines:  American  Scientists  and  Subsurface  Warfare  in  World  War  II,  National  Defense 
University  Press,  Washington,  DC,  1990. 

16  Fortun,  M  and  S.S.  Schweber,  "Scientists  and  the  Legacy  of  World  War  II:  The  Case  of  Operations  Research  (OR),"  Social  Studies  of 
Science,  Volume  23,  Number  4  (November  1993),  pp.  595-642. 
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mining  of  Japan's  Inland  Sea.  During  the  campaign,  more  than  1,000  ships  were 
damaged  or  sunk,  including  over  100  naval  vessels,  greatly  reducing  Japanese  shipping 
in  this  area.17'  18 

When  the  U.S.  entered  WWII,  the  loss  of  shipping  to  U-boats  in  the  Atlantic  rapidly 
reached  catastrophic  proportions.  In  March  1942,  Captain  Wilder  D.  Baker,  Commander 
of  Destroyer  Squadron  31  formed  the  Anti-Submarine  Warfare  Unit  in  Boston.  It  did  not 
take  the  Unit  long  to  conclude  that  the  major  cause  of  America's  poor  ASW  campaign 
was  inadequate  tactical  doctrine.  By  the  end  of  March,  Baker  approached  Professor 
Philip  M.  Morse  of  MIT  and  asked  him  to  undertake  the  role  of  lead  analyst  in  what 
would  be  called  the  ASW  Operations  Research  Group  (ASWORG).  One  of  the  first  issues 
ASWORG  tackled  was  the  problem  of  search,  by  ship  or  plane,  for  submarines  and 
warships.  Its  first  memorandum,  published  1  May  1942,  was  titled  "Preliminary  Report 
on  the  Submarine  Search  Problem."  The  recommendations  in  this  report  were 
immediately  put  into  effect  and  improved  the  tactics  of  convoy  protection  and  the 
search  for  U-boats.  As  successes  grew,  ASWORG  gradually  undertook  studies  in  the 
Pacific,  first  on  submarine  operations,  then  naval  air  activities,  and  finally  on  all  aspects 
of  operations.19,  20  During  this  time  Operations  Research  was  successfully  employed  in 
both  the  Army  and  Army  Air  Forces  as  well.21 

OR  and  the  Cold  War 

After  World  War  II,  acceptance  of  Operations  Research  techniques  became  widespread 
throughout  industry  leading  to  the  establishment  of  professional  societies  in  the  United 
States,  United  Kingdom,  and  Canada.  OR's  utility  to  military  operations  continued  with 
the  introduction  of  Federally  Funded  Research  and  Development  Centers  (FFRDCs),  first 
with  the  Center  for  Naval  Analyses,  then  Project  Air  Force,  and  then  followed  quickly  by 
Lincoln  Laboratories.22  At  the  outset  of  the  Cold  War,  Operations  Research  found  new 
missions,  most  notably  in  the  development  of  the  Single  Integrated  Operational  Plan 
(SIOP),  a  plan  and  planning  process  to  determine  the  use  of  U.S.  nuclear  weapons  in  the 
event  of  nuclear  war.  The  first  such  plan  was  submitted  in  1960.  Operations  Research 
tools  are  still  employed  to  study  blast,  thermal,  and  radiation  effects,  as  well  as  mission- 
asset  allocation.23  However,  since  1983  this  capability  has  diminished  significantly.24 


Shortley,  George,  "Operations  Research  in  Wartime  Naval  Mining,"  Operations  Research,  Volume  15,  Number  1  (January-February 
1967),  pp.  1-10. 

13  McCloskey,  Joseph  F.,  "U.S.  Operations  Research  in  World  War  II,"  Operations  Research,  Volume  35,  Number  6  (November  -  December 
1987),  pp.  910-925. 

19  Tidman,  Keith  R.,  The  Operations  Evaluation  Group:  A  History  of  Naval  Operations  Analysis,  Naval  Institute  Press,  Annapolis,  MD,  1984. 

20  Morse,  Philip  M.,  "The  Beginnings  of  Operations  Research  in  the  United  States,"  Operations  Research,  Volume  34,  Number  1  (January  - 
February  1986),  pp.  10-17. 

21  Shrader,  Charles  R.,  History  of  Operations  Research  in  the  United  States  Army,  Volume  1: 1942-1962,  United  States  Army,  Washington, 
DC,  2006. 

22  U.S.  Congress,  Office  of  Technology  Assessment,  A  History  of  the  Department  of  Defense  Federally  Funded  Research  and  Development 
Centers,  OTA-BP-ISS-157,  U.S.  Government  Printing  Office,  Washington,  DC,  1995. 

23  Pringle,  Peter  and  William  Arkin,  SIOP,  the  Secret  U.S.  Plan  for  Nuclear  War,  Norton  Publishing,  New  York,  New  York,  1983. 
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Also  in  the  early  1960s,  Secretary  of  Defense  Robert  McNamara,  convinced  that  the 
Secretary  of  Defense  should  control  the  evaluation  of  military  needs,  established  the 
Systems  Analysis  discipline  as  an  administrative  tool  to  estimate  the  required  quantity 
and  performance  of  forces  and  weapons  in  relation  to  their  mission  and  costs  known  as 
the  Planning,  Programming  and  Budgeting  System  (PPBS).25  The  effort  was  not  entirely 
successful  -  "McNamara's  systems  analysis  instilled  structure  and  discipline,  too,  but  in 
the  Pentagon,  it  was  applied  in  an  atmosphere  of  competition  and  distrust.  He 
introduced  systems  analysis  explicitly  to  obtain  control  over  Service  programs.  The 
mutual  trust  and  regard  in  the  fleet  were  impossible  in  Washington.  Decision  making  by 
analysis  took  on  a  hostile,  jaded,  and  onerous  cast."26 

Goldwater-Nichols  Act 

In  1986,  the  Goldwater-Nichols  Department  of  Defense  Reorganization  Act  called  on  the 
Joint  Chiefs  of  Staff  to  advocate  the  effectiveness  of  joint  operations.  In  so  doing,  it 
tasked  the  Vice  Chairman  to  chair  a  special  council  on  military  requirements  known  as 
the  Joint  Requirements  Oversight  Council  (JROC)  to  provide  programmatic 
recommendations  to  the  Chairman  of  the  Joint  Chiefs  for  consideration  and  submission 
to  the  Secretary  of  Defense.27  This  Act  positioned  the  Joint  Staff  to  be  the  only  military 
body  to  address  force  integration  and  to  resolve  inter-service  conflict  with  a  total 
organizational  perspective.  It  assigned  the  Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS) 
with  determining  strategy,  military  requirements,  programmatic  prioritization,  and 
budget  recommendations.  The  fact  that  the  Joint  Staff  is  comprised  of  military 
personnel  avoided  the  problems  encountered  with  the  McNamara  PPBS.28  In  1994, 
Admiral  William  Owens,  Vice  Chairman  of  the  JCS  established  the  Joint  Warfighting 
Capability  Assessment  (JWCA)  process  to  provide  better  advice  to  the  CJCS  by  increasing 
the  involvement  of  the  Combatant  Commanders  and  the  Services  in  the  JROC  process. 
The  JWCA  process  "examines  relationships  and  interactions  among  warfighting 
capabilities  and  identifies  opportunities  for  improving  joint  effectiveness."29  Each  team 
is  comprised  of  representatives  from  the  Unified  Commands,  OSD,  and  the  Services  and 
is  chartered  with  assessing  areas  where  capabilities  gaps  or  duplication  may  exist,  as 
well  as  determining  where  technologies  may  be  brought  to  bear.  The  JWCA  findings  are 
then  presented  to  the  JROC  for  consideration.  The  process  has  been  instrumental  in 
providing  a  deeper  analytic  underpinning  to  the  recommendations  from  the  CJCS  to  the 
Secretary. 


24  Technology  Strategies  &  Alliances,  Prepared  for  the  Office  of  Net  Assessment,  Maintaining  Advantage  in  a  Multi-polar  Nuclear  World: 
Declining  Resources  Final  Report,  December  2007. 

25  Enthoven,  Alain  and  K.  Wayne  Smith,  How  Much  is  Enough?:  Shaping  the  Defense  Program  1961-1969,  RAND  Corporation,  Santa 
Monica,  CA. 

26  Hughes,  Wayne  P.,  "Navy  Operations  Research,"  Operations  Research,  Volume  50,  Number  1  (January  -  February  2002),  pp.  103-111. 

27  Owens,  William  A.,  "JROC:  Harnessing  the  Revolution  in  Military  Affairs,"  Joint  Forces  Quarterly,  Summer  1994,  pp.  55-57. 

28  Turcotte,  Dr.  William  E.,  "Service  Rivalry  Overshadowed,"  Airpower  Journal,  Fall  1996,  pp.  28-33. 

29  Statement  of  General  Joseph  W.  Ralston,  The  Vice  Chairman  of  the  Joint  Chiefs  of  Staff,  to  the  Senate  Armed  Services  Committee,  10 
March  1999. 
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To  bring  the  historical  context  up  to  date,  there  are  two  Major  Acquisition  Program 
examples  in  which  OR  tools  were  used  effectively.  These  examples  may  be  instructive 
for  ISR  investment  decision  making. 

Joint  Direct  Attack  Munition  Program 

Operations  Research  techniques  were  the  key  element  in  the  process  that  led  to  the 
definition  of  the  requirements  and  optimization  of  the  trade  space  for  the  Joint  Direct 
Attack  Munition  (JDAM). 

Following  Desert  Storm,  a  review  of  lessons  learned  identified  the  fact  that  adverse 
weather  or  limited  visibility  of  the  ground  caused  by  blowing  sand,  smoke  from  ground 
fires,  fog  and  cloud  cover  limited  the  employment  of  precision  guided  weapons.  These 
same  conditions  also  limited  the  delivery  accuracy  of  unguided  weapons  when  delivered 
from  medium  and  high  altitudes. 

Data  from  Desert  Storm  was  analyzed  by  a  team  of  operators  and  requirements  officers 
from  the  Tactical  Air  Command  and  the  Strategic  Air  Command  (later  combined  into  Air 
Combat  Command).  This  process  was  facilitated  by  Glenn  Kent  of  the  RAND 
Corporation.  The  team  used  optimization  techniques  to  identify  the  needed  key 
characteristics  of  the  new  weapon  and  sensitivity  analysis  to  identify  the  available  trade 
space  during  acquisition. 

The  key  characteristics  included  reduced  sensitivity  to  weather,  reduced  crew  workload 
(laser  weapons  required  human-in-the-loop  through  most  of  the  flight  time),  launch  and 
leave,  and  reduced  cost  over  existing  precision  weapons.  Air  Force  Research  Laboratory 
personnel  then  joined  the  process  and  various  guidance  schemes  were  traded-off 
against  the  key  characteristics  before  arriving  at  a  guidance  scheme  which  featured  GPS- 
aided  inertial  guidance. 

The  end-game  set  of  requirements  and  technology  trades  defined  an  Adverse  Weather 
Precision-Guided  Weapon  which  became  the  JDAM  following  industry  competition. 

Joint  Strike  Fighter 

Operations  Research  methodologies  were  critical  to  the  Joint  Strike  Fighter  (JSF) 
Requirements  Process,  Technology  Development  Activity  and  affordability  trades.  The 
JSF  program  was  initiated  when  OSD  was  forced  to  cancel  three  separate  fighter 
development  activities  with  very  diverse  characteristics.  During  the  first  phase  of  the 
program,  called  the  Joint  Advanced  Strike  Technology  Program  (JASTP),  a  joint  process 
involving  service  war-fighters  and  technologists  was  employed  which  identified  the  key 
characteristics  of  a  next-generation  strike  system  for  the  USAF,  USN,  USMC  and  Royal 
Navy. 

Structured  analysis  of  the  needs  against  scenarios  defined  by  the  Joint  Staff  led  to  a  set 
of  key  characteristics,  associated  sensitivity  analysis,  and  a  Concept  of  Operations. 
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Industry  primes  then  entered  the  process  and  began  identifying  design  space  around 
the  key  characteristics.  System  design  attributes  were  then  analyzed  against  the  key 
characteristics  and  cost  and  displayed  in  carpet  plots  which  allowed  senior  Service  and 
OSD  decision  makers  to  understand  cost  as  an  independent  variable. 

This  process  resulted  in  the  Services  coalescing  around  a  set  of  requirements  that 
allowed  satisfaction  through  a  "Family  of  Airplanes"  concept  that  allowed  for  a  high 
degree  of  commonality  and  program  affordability. 

As  the  industry  prime  contractors  refined  their  design  concepts,  they  were  evaluated  by 
service  warfighters  and  maintainers  in  simulation  activities  conducted  by  the  Program 
Office.  This  simulation  used  several  core  tools  provided  by  and  maintained  by  the 
Program  Office.  This  provided  configuration  control  and  increased  confidence  in  the 
simulation  results. 

As  the  program  entered  the  System  Design  and  Development  (SDD)  phase,  this 
simulation  activity  and  the  carpet  plots  provided  decision  makers  the  tools  to  continue 
to  make  system  trades  for  program  affordability. 

3.1  General  Lessons  Learned  from  Historical  Experience 

There  are  two  important  lessons  to  be  learned  from  the  historical  examples  cited  above. 
The  first  lesson  is  that,  for  Operations  Research  to  have  a  positive  impact,  the  analyst 
should  be  imbedded  in  or  closely  connected  to  the  decision  maker's  organization.  In 
1941,  in  a  note  prepared  by  Professor  P.M.S.  Blackett  entitled  "Scientists  at  the 
Operational  Level,"  he  stated,  "The  object  of  having  scientists  in  close  touch  with 
operations  is  to  enable  operational  staffs  to  obtain  scientific  advice  on  those  matters 
which  are  not  handled  by  the  Service  technical  establishments."  In  addition, 
"Operational  staffs  provide  the  scientists  with  the  operational  outlook  and  data.  The 
scientists  apply  scientific  methods  of  analysis  to  these  data,  and  are  thus  able  to  give 
useful  advice."30 

While  outstanding  work  was  done  by  organizations  on  both  sides  of  the  Atlantic  during 
World  War  I,  they  were  never  accepted  by  the  Services  they  were  established  to  assist. 
In  the  case  of  the  Naval  Consulting  Board,  they  failed  to  have  any  great  effect  on  the 
conduct  of  naval  operations  because  they  were  a  civilian  organization  that  reported 
directly  to  the  Secretary  of  the  Navy.  Not  once  throughout  the  course  of  the  War  did 
the  Board  establish  contact  with  the  operational  commanders  in  the  field.31 

The  widespread  success  of  operations  research  in  World  War  II  can  be  attributed  to  the 
close  relationship  the  analyst  had  with  the  warfighter.  In  most  cases,  the  analyst  was 
deployed  to  support  a  specific  command  in  a  specific  theater  of  operations.  Data  were 
readily  available  at  the  source,  and  interactions  with  staff  personnel  enabled  the  analyst 


30  Air  Ministry,  The  Origins  and  Development  of  Operational  Research  in  the  Royal  Air  Force,  Her  Majesty's  Stationary  Office,  London,  1963. 

31  Whitmore,  W.F.,  "Edison  and  Operations  Research,"  Operations  Research,  Volume  1,  Issue  2  (February  1953),  pp.  83-85. 
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to  personally  demonstrate  the  utility  of  operations  analysis  through  near-immediate 
results. 

During  the  1960s,  McNamara's  "Whiz  Kids"  failed  to  take  notice  of  the  successes 
enjoyed  by  Operations  Research  groups  during  World  War  II  as  a  result  of  their  close 
alignment  with  the  decision  maker  -  an  alignment  not  present  with  operators  -  thereby 
alienating  the  Services.  The  result  was  Service  "stiff-arming"  of  the  PPBS  process  and  its 
ultimate  failure.  With  the  JROC  and  JWCA  processes,  control  of  the  analytic  process, 
regardless  of  its  rigor,  has  been  returned  to  the  DoD. 

The  second  lesson  to  be  learned  is  that  Operations  Research  can  be  misused  if  the 
analysis  team  is  not  free  from  bias.  The  case  in  point  is  the  deliberate  use  of  analysis  to 
support  "Bomber"  Harris'  position  to  continue  area  bombing  of  German  cities  during 
World  War  II  rather  than  redirecting  the  forces  at  his  disposal  to  conduct  precision 
bombing  of  military  targets  (airfields,  shipyards,  U-boat  pens,  etc.).  As  early  as  the 
autumn  of  1944,  Strategic  Bombing  Surveys  were  initiated  by  both  the  United  States 
and  Britain.  These  surveys  revealed  that  neither  civilian  morale  nor  the  German 
economy  were  as  severely  impacted  as  the  Bomber  Command  Operational  Research 
Section  (ORS)  claimed.  In  fact,  it  is  interesting  to  note  that  British  analysts  stated  that 
the  effects  of  the  London  Blitz  were  not  long-lasting.  Analyses  conducted  by  the  ORS 
were  biased  because  the  analysts  were  influenced  by  Harris'  strong  personality  and  the 
refusal  to  depart  from  pre-war  concepts  derived  from  the  Japanese  attack  on  Shanghai 
in  1931  and  the  obliteration  of  Guernica  during  the  Spanish  Civil  War  by  the  German 
Condor  Legion. 
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Chapter  4:  Relevance  of  Operations  Research  to  Intelligence, 
Surveillance,  and  Reconnaissance 

The  Task  Force  was  tasked  by  the  USD(I)  to  examine  how  OR  can  be  used  to  support  ISR 
decision  making.  Although  OR  may  have  wider  applicability  in  broader  investment 
decision  making,  the  focus  on  ISR  is  driven  by  several  factors  that  make  it  both  relevant 
and  challenging. 

A  more  consistent  scientific  and  quantitative  approach  to  evaluation  could  assist  the 
USD(I)  and  the  Director  of  National  Intelligence  (DNI)  with  many  key  investment  issues. 
Below  are  a  few  examples  of  past  and  current  issues  that  illustrate  the  trade-offs  that 
can  be  illuminated  by  the  proper  application  of  OR. 

4.1  Current  ISR  Issues  that  can  Benefit  from  the  Application  of  OR 

In  the  current  conflicts,  full  motion  video  (FMV)  collection  has  proven  to  be  a  very 
valuable  intelligence  input.  Flowever,  studies  suggest  that  a  large  number  of  UASs, 
bandwidth  and  exploitation  resources  would  be  required  to  meet  all  the  anticipated 
requirements  for  FMV.  A  program  known  as  the  Wide-Area  Airborne  Sensors  is 
designed  to  collect  a  wider  field  persistent  view  of  the  battle  space  as  a  way  of  capturing 
information.  Understanding  the  cost/benefit  of  these  programs  and  their  optimum  mix 
would  be  very  valuable  at  this  juncture. 

With  the  focus  on  current  operations,  resources  allocated  to  science  and  technology 
(S&T)  collection  and  exploitation  have  been  reduced.  S&T  data  is  essential  for 
identifying  new  adversary  capabilities  and  determining  approaches  to  collect  against 
them.  The  lack  of  S&T  data  creates  vulnerabilities  in  the  United  States'  ability  to 
counter  new  threats  and  collect  information  against  adversaries  when  they  are  using 
new  technologies.  The  key  to  making  acquisition  decisions  is  understanding  the 
appropriate  balance  between  S&T  and  current  collection  capabilities  to  ensure  a 
continued  ability  to  adapt  and  understand  new  threats.  Acquisition  delays  and  failing 
national  systems  programs  require  key  investment  decisions.  These  decisions  weigh 
trading  lower  risk  and  lower  capability  short  term  solutions  against  higher  risk  and 
higher  capability  longer  term  solutions.  Prudent  investment  decisions  should  be  based 
on  the  optimization  of  factors  such  as  how  UASs  and  manned  aircraft  can  work  with 
national  systems  to  provide  support  to  national  and  military  needs  over  the  next  10 
years. 

In  the  Global  War  on  Terror,  information  about  individuals  can  be  more  important  than 
information  on  military  equipment.  This  has  given  rise  to  the  desire  to  collect  and  use 
biometric  data  on  the  battlefield.  Flowever,  assessing  the  value  of  this  information  and 
how  it  will  be  used  versus  the  value  of  other  kinds  of  collection  will  be  important.  In 
addition,  as  investments  in  biometrics  increase,  OR  should  prove  helpful  in  ensuring  that 
investments  are  made  in  a  balanced  fashion  across  the  entire  intelligence  cycle  (i.e., 
requirements,  tasking,  collection,  processing,  exploitation,  dissemination). 
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4.2  Past  ISR  Investment  Issues  that  have  Benefited  from  the  Application  of  OR 

The  requirements  process  for  the  Future  Imagery  Architecture  (FIA),  the  Integrated 

Overhead  SIGINT  Architecture  (IOSA),  the  recent  U-2  / 
Global  Flawk  trade-off  decision,  and  the  aforementioned 
TH E  SUCCESSFUL  USE  OF  HVT  Study  employed  OR  in  a  significant  manner.  In  all  of 

OPERATIONS  RESEARCH  these  cases,  there  was  a  clear  mandate  from  decision 

REQUIRES  SOUND  makers  for  the  use  of  OR,  general  consensus  in  advance  on 

ASSUMPTIONS  the  objective  function,  a  broad  base  of  community 

involvement,  and  sound  methodology.  The  Task  Force 
notes,  however,  that  the  use  of  OR  cannot  guarantee 
successful  program  implementation,  particularly  if  technical  capabilities  for 
implementation  assumed  to  exist  during  analysis  are  not  present  during  design  and 
development.  As  always,  the  successful  use  of  OR  requires  sound  assumptions. 

4.3  Complications  in  Applying  OR  to  ISR  Investment  Issues 

ISR  programs  are  complicated  by  several  factors  that  make  investment  decision  making 
difficult. 

To  add  value  for  a  decision  maker,  information  has  to  be  collected,  processed,  exploited 
and  disseminated.  Typically  the  programs  that  operate  in  each  of  these  stages  are 
controlled  by  different  organizations.  Thus,  in  evaluating  an  investment  decision  and  its 
impacts,  the  OR  analyst  must  be  careful  to  look  across  program  elements  and  across 
organizations.  In  many  cases,  the  funding  streams  to  support  a  program  may  include 
both  Military  Intelligence  Program  funding  and  National  Intelligence  Program  funding. 
The  first  is  controlled  by  the  Secretary  of  Defense  under  Title  10  authority,  the  latter  by 
the  DNI  under  Title  50  authority. 

Most  ISR  systems  are  used  to  support  a  broad  variety  of  missions.  For  example,  the 
same  UAS  may  be  tasked  to  support  force  protection,  battlespace  awareness,  and 
targeting.  National  systems  are  often  used  to  support  decisions  ranging  from  national 
strategy  and  policy  to  striking  individual  tactical  targets.  Evaluating  the  value  of  a 
system  requires  looking  at  its  impact  on  all  of  the  missions  it  might  support. 

Most  major  ISR  systems  are  complex  and  expensive,  which  generally  results  in  both  a 
long  development  time  and  a  long  design  life.  These  factors  often  result  in  an  ISR 
system  operating  for  more  than  20  years  after  the  requirements  for  the  system  were 
initially  developed.  During  that  time  period,  threats  may  change  substantially  and  the 
types  of  uses  for  which  the  system  was  initially  designed  may  become  less  relevant.  To 
make  effective  investment  decisions  throughout  the  life  cycle  of  ISR  assets  requires  a 
broad  view  of  what  the  future  may  hold  so  that  the  system  can  be  evaluated  against  all 
potential  future  uses. 
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4.4  Challenges  in  Applying  OR  to  ISR  Issues 

The  proper  use  of  OR  to  examine  ISR  investment  decisions  objectively  can  address  many 
of  the  complications  identified  above.  However,  applying  OR  to  ISR  issues  has  several 
key  challenges.  The  primary  challenge  is  to  answer  the  question:  "What  is  a  pound  of 
ISR  worth?"32  That  is,  what  is  an  objective  measure  of  the  value  of  ISR  information? 
Unlike  lethal  weapon  systems,  ISR  does  not  directly  destroy  enemy  targets.  Instead,  it 
produces  knowledge  that  informs  decision  makers  as  they  make  a  broad  range  of 
decisions.  This  issue  makes  it  difficult  to  agree  on  the  objective  function  against  which 
OR  is  performed.  To  evaluate  what  this  knowledge  is  worth  requires  some 
understanding  of  how  decisions  are  made  and  what  contribution  ISR  provides  to  those 
decisions.  Although  there  are  no  universally-accepted  measures  that  have  been 
developed  to  address  this  issue,  there  are  some  approaches  that  have  been  used  to 
measure  the  impact  of  a  specific  ISR  contribution  in  specific  circumstances. 

The  most  common  ISR  metric  used  for  national  systems  evaluation  is  "User 
Satisfaction."  User  Satisfaction  allows  analysts  to  quantify  the  impact  of  an  ISR  system 
by  measuring  the  fraction  of  collection  requirements  it  is  able  to  meet.  Using  this 
metric,  trade-off  decisions  can  be  made  between  architectures  or  systems  by  assessing 
the  amount  of  "User  Satisfaction"  they  produce  versus  the  cost  of  the  system. 
Unfortunately,  this  approach  is  heavily  dependent  on  the  validity  of  the  requirements 
provided,  and  it  assesses  only  the  relative  value  -  not  the  absolute  value  -  of  the 
systems  being  traded.  That  is,  it  does  not  identify  how  much  is  enough  or  the  direct 
impact  of  this  information  on  outcomes.  It  is  also  hard  to  evaluate  systems  from 
different  collection  disciplines,  since  user  requirements  are  generally  specified  by 
discipline. 

Other  OR  approaches  attempt  to  characterize  the  effect  of  ISR  on  outcomes  by  trying  to 
quantify  the  effect  of  ISR  on  attrition  of  enemy  forces,  a  traditional  measure  of 
effectiveness.  There  have  been  many  different  ways  of  addressing  this  challenge,  with 
most  suffering  from  strong  limitations.  Typically  the  tactical  effect  of  finding,  classifying, 
and  locating  a  target  can  be  related  to  the  rate  and  probability  of  killing  the  target. 
These  "sensor-to-shooter"  models  capture  some  of  the  tactical  impacts  of  ISR,  but  do 
not  capture  the  value  of  situational  awareness  or  the  value  of  ISR  in  non-combat 
situations. 

Attempts  to  capture  the  wider  value  of  ISR  are  often  stymied  by  the  view  held  by  many 
practitioners  that  ISR  is  a  craft  that  is  too  complex  to  be  characterized.  This  view  makes 
it  difficult  to  address  quantitative  measures  for  system  performance.  As  a  result,  the 
community  will  have  to  continue  to  study  the  impact  of  ISR  on  decision  making  and 
outcomes  to  better  characterize  its  value.  The  development  of  community  accepted 
metrics  and  the  agreement  by  decision  makers  on  objective  functions  in  advance  of  an 
OR  effort  is  essential  to  the  successful  application  of  OR. 


32  As  quoted  by  the  Honorable  James  R.  Clapper,  Jr.,  Under  Secretary  of  Defense  for  Intelligence. 


Operations  Research  Applications  for  ISR 


23 


Chapter  4:  Relevance  of  Operations  Research  to  ISR 


This  page  is  intentionally  blank. 


24 


Defense  Science  Board  Advisory  Group  on  Defense  Intelligence 


_ Chapter  5:  Current  Operations  Research  Practices  ( TOR  Tasks  1  and  2) 

Chapter  5:  Current  Operations  Research  Practices  (TOR  Tasks  1  and  2) 

Given  the  Task  Force's  understanding  of  OR,  its  attributes,  the  conditions  under  which  it 
can  be  employed  successfully,  and  in  response  to  TOR  Tasks  1  and  2,  the  Task  Force 
looked  at  several  specific  examples  of  OR  from  a  broad  array  of  organizations  across  the 
Department  of  Defense,  the  Intelligence  Community  (1C),  and  industry.  Given  the  time 
available  to  the  Task  Force,  it  was  not  possible  to  explore  the  entirety  of  OR  and  OR- 
related  activities  in  the  Department  and  the  1C.  Flowever,  the  representative  sampling, 
while  not  exhaustive,  provided  the  Task  Force  with  a  good  opportunity  to  assess  the 
extent  to  which  OR  is  institutionalized,  how  it  is  being  employed,  and  how  its 
employment  might  be  improved.  This  approach  should  allow  the  USD(I)  to  infer  the 
extent  to  which  OR  is  being  used  to  support  ISR  investment  decisions.  A  list  of 
organizations  that  made  presentations  is  located  in  Appendix  C. 

The  Task  Force  received  presentations  from  entities  from  each  of  the  Services,  from  the 
National  Geospatial-Intelligence  Agency  (NGA),  the  National  Security  Agency  (NSA),  the 
National  Reconnaissance  Office  (NRO),  ODNI,  OSD  PA&E,  and  others,  including  industry. 
Edge  Consulting,  Inc.  provided  a  briefing  on  their  ongoing  work  regarding  ISR  in  support 
of  the  search  for  High  Value  Targets.  The  Navy  has  pockets  of  excellence  where  OR  is 
conducted,  and  most  importantly  has  a  strong  curriculum  for  OR  at  the  Naval 
Postgraduate  School  (NPS).  The  Army,  which  has  made  the  most  consistent 

commitment  to  institutionalizing  OR,  uses  the  NPS 
to  great  advantage  to  educate  their  OR  workforce. 

The  Air  Force  also  maintains  an  OR  degree  program 
at  the  Air  Force  Institute  of  Technology  (AFIT).  The 
most  robust  OR  at  the  OSD  level  is  performed  by 
PA&E.  Of  note,  it  is  now  almost  exclusively 
outsourced  with  little  internal  capability.  The 
Intelligence  Community  brought  multiple  examples 
to  the  Task  Force  of  the  ongoing  decline  in  OR 
capability  in  support  of  planning,  programming,  and 
materiel  acquisition.  FedEx  presented  a  compelling  example  of  a  model  used  in  the 
private  sector  for  OR  institutionalization  and  application.  Finally,  the  expertise 
embodied  in  the  MORS,  a  professional  organization  dedicated  to  defense-related  OR, 
represents  a  unique  resource  for  the  Department. 

From  an  overarching  perspective,  there  are  some  findings  that  became  clear  to  the  Task 
Force.  While  OR  is  used  widely  across  the  Services,  especially  in  support  of  operations, 
the  application  of  OR  is  generally  uncoordinated,  sometimes  organizationally  biased, 
inconsistently  resourced,  and  not  well  integrated  into  processes  and  programs.  There  is 
no  strong  central  champion  for  OR  at  the  Department  level,  and  therefore  there  is  no 
clear  ability  to  identify  and  support  best  practice  transfer,  or  to  capture,  document,  or 
share  OR  lessons  learned.  Consequently  there  is  not  a  jointly  accepted  OR  review  or 
accrediting  organization.  While  Federal  Acquisition  Regulations  require  OR  for  Major 


...THE  APPLICATION  OF  OR  IS 
GENERALLY  UNCOORDINATED, 
SOMETIMES  ORGANIZATIONALLY 
BIASED,  INCONSISTENTLY 
RESOURCED,  AND  NOT  WELL 
INTEGRATED... 
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Acquisition  Programs,  the  application  of  OR  in  response  to  this  requirement  is 
inconsistent  in  level  of  effort  and  quality,  and  often  requires  rework  when  submitted  for 
OSD  review. 

Efforts  to  support  and  resource  OR  have  generally  been  in  decline  in  the  Department, 
but  this  trend  is  even  more  stark  in  the  Intelligence  Community.  In  the  cases  of  the 
NGA,  NSA,  and  the  NRO,  it  was  clear  that  OR  capabilities  for  programming  and 
acquisition  decision  making  are  atrophying  rapidly.  The  Task  Force  did  note  that  in 
contrast,  these  same  agencies  use  significant  and  effective  OR  in  support  of  current 
operations.  Notably,  there  is  a  nascent  analytic  capability  in  the  Defense  Intelligence 
Operations  Coordination  Center  (DIOCC),  and  while  its  parent  organization,  the  National 
Intelligence  Coordination  Center  (NIC-C)  was  originally  intended  to  have  a  strong  OR 
capability,  that  effort  has  not  been  resourced.  Finally,  there  is  no  evidence  of  OR 
leadership  or  direction  from  the  staff  of  the  Director  of  National  Intelligence,  nor  any 
significant  OR  capability  noted  on  that  staff.  The  recent  decision  to  terminate  the 
Integrated  Concepts  Development  Office  (ICDO)  is  further  evidence  of  the  apparent 
decline  of  interest  in  OR  at  the  ODNI. 

Of  the  many  presentations  given  to  the  Task  Force,  two  stood  out  in  a  positive  manner. 
First,  the  Army  has  continued  to  support  a  historically  strong  OR  program.  The  Army 
has  developed  and  sustained  an  officer  career  field  for  OR  which  includes  opportunities 
for  advanced  degrees  and  promotion.  The  Army  has  continued  to  support  institutions 
that  promulgate  OR  best  practices,  including  the  Army  Training  and  Doctrine  Command 
(TRADOC)  Analyses  Center  (TRAC).  The  Army  has  traditionally  maintained  a  strong 
champion  on  the  Army  Staff  with  direct  access  to  the  most  senior  leadership.  The  Army 
uses  OR  both  in  support  to  current  operations,  (with  deployed  OR  specialists  embedded 
with  units  in  theater)  as  well  as  to  support  strategic  planning,  programming,  materiel 
acquisition,  and  the  generation  of  requirements. 

Second,  a  model  which  may  have  direct  applicability  to  the  Department  is  the 
application  of  OR  in  the  private  sector,  as  evidenced  by  FedEx.  FedEx  has  a  strong  and 
consistent  commitment  to  OR,  and  uses  OR  extensively  in  its  decision  making,  both  for 
daily  operations  and  long-term  strategic  planning.  The  FedEx  corporate  model  is  a  very 
small  (approximately  200  people)  headquarters  overseeing  multiple  robust  independent 
operating  units.  FedEx  has  a  hybrid  model  for  OR  which  is  both  centralized  and 
distributed.  A  small  OR  group  of  approximately  eight  to  ten  people  provides  corporate- 
level  OR  support,  and  each  of  the  line  operations  maintains  their  own  OR  entities.  OR  is 
valued  as  part  of  the  culture  at  all  levels  of  the  organization,  but  also  seen  through  a 
pragmatic  lens.  While  OR  is  often  used  to  great  advantage,  OR  entities  within  FedEx 
have  no  line  authority.  The  leader  of  the  corporate-level  OR  organization  made  it  clear 
that  in  no  case  would  OR  be  utilized  if  there  was  not  clear  leadership  support,  and 
articulated  that  even  the  most  well-executed  OR  would  be  of  no  value  if  the  decision 
maker  did  not  support  it. 
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FedEx  uses  OR  at  the  corporate  level  to  determine  which  lines  of  business  are 
appropriate,  how  much  capability  is  required  to  support  each  line  of  business,  and  what 
FedEx  should  be  prepared  to  invest  in  support  of  those  lines  of  business.  The  FedEx 
corporate  OR  organization  does  not  provide  decision  support  at  the  operational  level 
within  the  separate  lines  of  business,  but  does  directly  support  decision  making  for  the 
C-level  (CEO,  CFO,  CIO,  etc.)  corporate  leadership,  and  can  assist  the  C-level  leaders  with 
OR-related  expertise  as  they  review  the  analytics  submitted  by  the  operating  units.  The 
corporate  OR  entity  also  provides  some  measure  of  oversight  to  the  practice  of  OR  in 
the  operating  units  to  ensure  quality  and  consistency.  Within  the  line  operating  units, 
OR  is  used  to  optimize  efficiency,  to  test  alternative  operating  concepts,  to  guide 

investment  priorities,  and  to  support  other 
decisions  as  management  deems  necessary. 


THERE  IS  NO  CHAMPION  ADVOCATE 
IN  THE  DEPARTMENT,  NO  COHERNT 
STRATEGY,  AND  NO  ORGANIZATION 
OR  STAFF  ELEMENT  IDENTIFIED  TO 
PROVIDE  LEADERSHIP  IN  THIS 
DOMAIN. 


It  was  clear  to  the  Task  Force  that  OR  is  used,  in 
some  form  or  another,  in  every  Service  and  the 
intelligence  Combat  Support  Agencies.  It  was  also 
clear  that  save  for  a  few  "points  of  light,"  OR  is  in 
decline  and  not  universally  valued  in  decision 
making,  especially  at  the  strategic  level,  and  with 
respect  to  investment  decisions  and  systems 
acquisition.  There  is  no  champion  advocate  in  the 
Department,  no  coherent  strategy,  and  no  organization  or  staff  element  identified  to 
provide  leadership  in  this  domain.  Moreover,  while  the  1C  conducts  truly  world  class  OR 
in  determining  the  daily  allocation  of  scarce  collection  resources  against  myriad 
increasing  requirements,  there  has  been  an  astounding  decay  in  the  ability  of  these 
same  agencies  to  conduct  OR  in  support  of  planning,  programming,  and  acquisition.  In 
the  case  of  both  the  Army  and  FedEx,  however,  the  Task  Force  identified  some 
significant  best  practices  that  may  have  immediate  use  as  models  for  a  way  ahead  in  OR 
to  support  ISR  investment  decisions. 
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In  approaching  TOR  Task  3,  which  is  to  recommend  an  ISR  test  case  in  which  OR  might 
be  used  to  reach  a  better  decision,  the  Task  Force  found  it  useful  to  examine  the 
attributes  that  possible  test  cases  should  possess.  It  is  unlikely  that  every  possible  test 
case  will  be  characterized  by  every  attribute  discussed  herein,  but  the  Task  Force  is 
persuaded  that  successful  cases  should  be  characterized  by  all  or  most  elements 
described  in  the  sections  below. 

The  Task  Force  considered  several  test  cases  and  believes  two  cases  in  particular  would 
serve  the  USD(I)  in  establishing  ways  to  employ  OR  effectively  in  support  of  ISR.  These 
cases  are: 

•  Biometrics  technology  investment  and  architecture 

•  Balance  intelligence  cycle  investment 

These  test  cases  are  discussed  in  more  detail  in  Appendix  D. 

6.1  Importance 

The  Task  Force  considers  it  critical  that  potential  test  cases  be  viewed  by  ISR  decision 
makers  as  involving  issues  of  great  importance  -  certainly  to  the  ISR  community  and 
especially  to  USD(I)  and  DNI  leadership.  If  the  issue(s)  chosen  to  examine  with  OR  do 
not  meet  the  common  sense  test  of  importance,  they  are  unlikely  to  receive  the  kind  of 
leadership  "buy-in"  and  resource  support  that  will  be  required  for  success.  While  it  is 
true  that  the  Task  Force  believes  that  OR  could  significantly  contribute  to  ISR  decisions 
for  a  multitude  of  operational,  architectural,  and  investment  or  acquisition  issues,  we 
believe  that  it  is  the  latter  area  (investment/acquisition  of  significant  resources)  that  will 
provide  the  most  unique  benefit  to  the  USD(I)  and  DNI  missions. 

6.2  Leadership  Commitment 

Closely  related  to,  and  even  driven  by,  the  importance  attribute  described  above,  is  the 
issue  of  leadership  commitment.  If  any  selected  test  case  is  to  successfully  demonstrate 
the  value  of  OR  in  ISR  decision  making,  USD(I)  (and  probably  DNI)  leadership  must  be 
fully  committed  to  the  case  and  the  process.  The  following  elements  of  leadership 
commitment  to  the  test  case  must  be  present  and  evident: 

1.  Leadership  must  ensure  that  stakeholders  agree  to  the  terms  of  reference 
and  assumptions  for  the  test  case.  It  is  critical  that  the  questions  to  be 
answered  and  assumptions  to  be  used  are  clear  and  properly  framed. 

2.  The  test  case  sponsor  must  commit  to  providing  the  resources  necessary  to 
seeing  the  test  case  through  to  completion,  and  should  provide  a  "time- 
certain"  date  when  the  project  is  to  be  completed  if  it  is  to  have  value  in 
decision  making. 
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3.  The  test  case  sponsor  must  commit  to  using  the  OR  results  in  support  of  the 
decision.  While  it  is  not  realistic  to  believe  that  an  important  ISR  investment 
decision  will  be  driven  solely  by  the  OR  analytical  results,  it  should  be  clear  to 
stake-holders  that  the  results  were  carefully  considered  in  the  decision¬ 
making  process.  This  is  particularly  true  if,  for  some  reason,  the  ultimate 
decision  diverges  significantly  from  the  direction  suggested  by  the  OR 
analysis,  and  it  would  be  helpful  if  the  decision  maker  provided  feedback  as 
to  how  the  OR  analysis  was  used  and  the  degree  to  which  it  was  helpful 
and/or  could  have  been  more  useful. 

6.3  OR  Process  Credibility 

All  decision  makers  and  stake-holders  involved  with  the  chosen  test  case  must  be 
convinced  of  the  credibility  and  integrity  of  the  OR  process,  even  if  they  ultimately  might 
disagree  with  the  results.  This  means  that  everything  used  in  the  test  case  OR  process 
must  be  credible  and  transparent  -  terms  of  reference,  questions  to  be  answered,  data 
to  be  used,  OR  tools  to  be  applied,  OR  practitioners  to  be  utilized,  and  the  decision(s) 
intended  to  be  supported  by  the  process.  In  addition,  decision  makers  and  stake¬ 
holders  must  agree  in  advance  on  the  objective  functions  against  which  the  analysis  is 
conducted.  If  the  initial  test  case(s)  conducted  do  not  "ooze"  credibility,  the  prospects 
for  the  institutionalization  of  OR  for  making  ISR  investment  decisions  are  slim. 

6.4  Test  Case  Scope 

The  issue  of  test  case  scope  is  a  challenge.  The  Task  Force  has  already  highlighted  the 
attribute  of  importance,  but  to  be  important,  the  test  case  does  not  have  to  be  "huge"  - 
especially  in  terms  of  scope.  The  broader  the  scope  the  more  difficult  it  will  be  to 
achieve  stakeholder  "buy-in"  on  the  credibility  of  the  process  and  its  contribution  to 
good  decision  making.  In  this  regard,  an  optimal  test  case  may  be  one  that  employs  OR 
to  examine  the  potential  impacts  of  decisions  that  are  already  being  contemplated  or 
nearing  "finality,"  which  by  definition  are  likely  to  meet  the  "importance"  test  already 
described.  Generic  examples  of  these  kinds  of  "impact"  examinations  are  provided 
below: 

1.  Examining  the  impact  of  a  contemplated  ISR  investment  decision  on  its 
applicability  against  a  range  of  threats,  current  and  future.  For  example, 
assume  that  a  costly  investment  in  additional  high  altitude/long  endurance 
UASs  equipped  for  imagery  and  full  motion  video  is  contemplated  primarily 
for  GWOT,  and  specifically  for  OIF  and  OEF,  and  that  the  coverage  factor  and 
output  of  those  UASs  is  credibly  quantified  and  modeled.  An  OR  examination 
of  the  contemplated  investment  could  be  conducted  in  other  scenarios  -  for 
example  a  "defense  of  Taiwan"  or  "conflict  with  Iran."  In  this  example,  the 
need  for  the  UASs  for  GWOT  may  be  the  overriding  requirement,  but  one 
could  definitively  analyze  and  understand  in  a  credible  OR  process  whether 
the  same  UASs  equipped  with  the  same  sensors  will  be  applicable  to  the 


30 


Defense  Science  Board  Advisory  Group  on  Defense  Intelligence 


Chapter  6:  ISR  Test Case(s)  Attributes  (TOR  Task 3) 


other  scenarios  and  consider  future  alternate  utility  (either  positive  or 
negative)  into  the  investment  decision. 

2.  Examining  a  contemplated  ISR  investment  decision  in  a  specific  new 
discipline  for  its  potential  impact  (specifically  cost  and  capability)  on  other 
parts  of  the  relevant  intelligence  cycle.  For  example,  consider  the  impact  of 
a  costly  investment  in  biometric  data  collection  capability.  OR  techniques 
could  be  used  to  analyze  the  capability  of  the  entire  ISR  system  to  effectively 
use  the  contemplated  new  collection  capability  and/or  understand  what 
additional  costs  would  have  to  be  incurred.  An  obvious  case  in  point  would 
be  whether  or  not  appropriate  investment  has  been  made  in  the  analytical 
resources  (specific  skills,  recruitment  and  training)  and  dissemination 
capability  needed  to  handle  the  volume  of  new  product  that  would  be 
produced  by  a  UAS  investment.  Additionally,  in  an  area  like  biometrics,  are 
there  legal  or  privacy  impediments  that  make  the  collection  capability  more 
difficult  or  more  expensive  to  direct,  task,  and  use,  and  have  the  costs  of 
these  factors  been  credibly  analyzed  and  considered? 

3.  Examining  the  impact  of  new  war-fighting  capabilities  on  the  ability  of  the 
ISR  system  to  support  them.  A  notable  historic  instance  of  this  was  the 
impact  on  the  ISR  system  of  fielding  large  numbers  of  precision  munitions 
since  Desert  Storm  and  the  continuing  impact  of  fielding  "extreme  precision" 
systems  today.  For  example,  GPS-guided  weapons  are  only  as  precise  as  the 
quality  of  the  Digital  Terrain  Elevation  Data  (DTED)  that  is  collected  and 
analyzed  to  provide  exact  target  coordinates.  The  demand  for 
extraordinarily  detailed  and  high-quality  imagery  to  support  precision 
application  of  military  capability  (strike  or  SOF)  continues  to  grow 
significantly.  Can  OR  be  used  to  understand  and  plan  for  new  weapons  or 
strike  capabilities  on  the  horizon  that  will  produce  new  quality  or  volume 
demands  on  the  ISR  system?  One  possible  example  of  this  opportunity 
would  be  the  extension  of  precision  strike  into  ground  and  naval  force 
artillery.  The  dramatic  increase  in  precision  strike  potential  brought  on  by 
such  an  exponential  increase  in  precision  weapons  would  potentially  bring 
about  a  similar  exponential  increase  in  the  need  for  highly  accurate  target 
geographic  coordinates  and/or  high  quality  target  imagery.  OR  techniques 
seem  ideally  suited  to  examine  this  critical  question. 

4.  Examining  different  intelligence  disciplines/portions  of  the  intelligence 
cycle  to  determine  where  -  for  a  given  problem  -  investment  of  limited 
resources  can  provide  the  most  effective  and  efficient  prospects  for  mission 
satisfaction.  Since  this  type  of  examination  is  often  hindered  by  the  fact  that 
resource  sponsors  or  political  proponents  are  aligned  parochially  by 
intelligence  discipline,  organizational  interests,  budget  authorities,  or 
constituent  concerns  versus  mission  optimization;  unbiased  OR  using  good 
quality  data  and  credible  analytical  techniques  could  make  a  significant 
contribution.  Even  if  the  challenges  were  such  that  OR  did  not  "drive  the 
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decision,"  it  is  still  possible  that  applying  good  OR  techniques  could  improve 
the  quality  of  the  debate  and  create  the  environment  for  a  more  favorable 
outcome.  Examples  of  the  kind  of  problems  are  numerous:  HUMINT  vs. 
SIGINT  as  the  route  toward  discovering  enemy  intentions,  faster/broader 
dissemination  vs.  more  imagery  collection,  and/or  the  value  of  more 
investment  in  analysis  vs.  just  adding  collection  capability. 

6.5  Test  Case  Timing  and  Duration 

While  not  as  critical  as  the  other  attributes,  the  Task  Force  considers  it  optimal  if  the 
first  test  case(s)  could  begin  soon,  address  a  near  term  issue,  and  be  completed 
relatively  quickly.  It  will  obviously  create  more  1C  interest  if  the  OR  process  dealt  with 
issue(s)  that  are  currently  important  and  relevant  to  ongoing  resource  or  operational 
deliberations.  Similarly,  if  the  first  test  case(s)  were  suitable  for  a  credible  OR  analysis 
that  could  be  completed  in  three  to  six  months  it  would  be  helpful  in  getting  an  early 
determination  of  the  value  of  the  OR  process  for  ISR  decision  support. 

6.6  A  Note  on  ISR  Test  Case  Challenges 

While  the  Task  Force  believes  that  the  ISR  test  case  attributes  cited  above  are  valid,  it  is 
under  no  illusion  about  the  difficulties  of  applying  OR  in  the  ISR  community.  OR  is  hard 
for  ISR  and  challenges  abound: 

1.  Typically,  models  exist  primarily  for  collection  systems,  making  the  "SR" 
(surveillance  and  reconnaissance)  part  of  the  ISR  acronym  much  easier  to 
model  and  measure  than  the  "I"  (Intelligence).  In  fact,  probably  the  single 
biggest  challenge  for  using  OR  for  ISR  investment  decision  support  is  the  fact 
that  we  have  never  been  able  to  successfully  measure  or  model  the  value  of 
the  product  of  the  intelligence  process  -  knowledge  about  an  enemy's 
capabilities  and  intent.  In  short,  the  impact  of  intelligence  on  decision 
making  is  not  easily  measured  or  understood. 

2.  ISR  platforms  are  often  multi-purpose,  making  their  total  output  and  relative 
value  difficult  to  measure,  model,  and  compare. 

3.  The  ISR  community  lacks  analytic  models  that  reflect  the  utility  of  ISR  product 
assessment. 

4.  Funding  and  control  streams  are  in  different  organizations,  and  often  in 
different  departments  of  government. 

5.  The  same  ISR  platforms  and  agencies  often  serve  many  different  customers 
whose  requirements  range  from  tactical  through  strategic,  and  whose 
missions  can  range  from  combat  to  operational  planning  or  national  level 
policy. 
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Chapter  7:  Institutionalizing  Operations  Research  (TOR  Task  4) 

The  Task  Force  looked  at  a  number  of  current  OR  and  decision-support  institutions  in 
use  by  the  Services,  DoD,  the  1C,  and  industry.  The  Task  Force  found  that  OR,  systems 
analysis,  and  other  decision-support  disciplines  (including  modeling  and  simulation)  are 
used  widely  in  the  Services  and  DoD,  as  well  as  the  1C.  Flowever,  the  application  of  OR 
(and  related  disciplines)  is  uneven  in  these  organizations,  as  is  the  extent  to  which  OR  is 
recognized  as  a  distinct  discipline  and  career  field.  Some  organizations  (e.g.,  the  NRO) 
use  OR  to  optimize  the  use  of  current  ISR  resources  to  support  specific  mission 
requirements,  while  others  (e.g.,  the  NSA)  employ  modeling  and  simulation  to  support 
architecture  analysis.  The  former  example  represents  robust  and  sustained  use  of  OR, 
albeit  without  strong  coupling  of  the  resources  of  OR  analysis  to  strategic  investment 
decision  making.  The  latter  example  typifies  a  situation  in  which  structured  decision 
making  has  fallen  into  disuse  and  in  which  its  practitioners  struggle  to  find  opportunities 
to  apply  their  discipline.  Of  note,  the  Army  has  done  the  most  to  sustain  an 
institutionalized  OR  capability,  one  that  is  well  instantiated  at  the  TRAC.  TRAC's  Website 
states: 


"TRAC  conducts  operations  research  (OR)  on  a  wide  range  of  military 
topics,  some  contemporary  but  most  often  set  5  to  15  years  in  the  future. 

How  should  Army  units  be  organized?  What  new  systems  should  be 
procured?  How  should  soldiers  and  commanders  be  trained?  What  are 
the  costs  and  benefits  of  competing  options?  What  are  the  potential  risks 
and  rewards  of  a  planned  military  course  of  action?  TRAC  directly 
supports  the  mission  of  the  Army's  major  command,  the  Training  and 
Doctrine  Command  (TRADOC),  to  develop  future  concepts  and 
requirements  while  also  serving  the  decision  needs  of  many  military 
clients.  TRAC  analysis  enables  change."33 

While  the  Task  Force  did  not  assess  in  detail  the  extent  to  which  TRAC  is  performing 
successfully  in  all  of  the  roles  described  above,  our  discussions  allowed  us  to  conclude 
that  TRAC  is  comprised  of  dedicated  professionals  who  enjoy  leadership  commitment 
and  support  and  whose  work  is  visible  and  highly  regarded  in  the  Army  planning  and 
operational  communities. 

As  described  in  Chapter  5,  the  FedEx  example  also  represents  a  seemingly  robust 
approach  to  institutionalizing  OR  and  structured  decision  making.  This  model  employs 
OR  at  a  corporate  level  to  help  decide  on  the  lines  of  business  in  which  the  company 
should  compete,  and  at  what  level.  FedEx  also  employs  OR  at  the  line-of-business  level 
to  support  investment  decision  making  and  to  configure  operational  resources;  it 
provides  corporate  oversight  to  ensure  the  presence  of  consistent  OR  standard 
processes  and  methodology  throughout  the  FedEx  family  of  businesses.  The  company 
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does  not  "require"  the  use  of  OR  so  much  as  the  leadership  commitment  of  the 
company  to  OR-based  decision  making  gives  its  practitioners  substantial  influence  as 
high  value  decisions  are  made.  That  leadership  extends  to  empowering  the  company's 
corporate  OR  organizational  to  ensure  that  appropriate  professional  standards, 
processes,  and  practices  are  employed  in  the  conduct  of  OR  throughout  the  company's 
various  lines-of-business. 

Four  common  elements  unite  the  Army  and  FedEx  cases. 

•  The  OR  professionals  in  both  organizations  enjoy  leadership  commitment 
and  support.  In  both  organizations,  senior  leaders  employ  the  results  of  OR 
in  a  visible  fashion  to  support  key  strategic  decisions.  In  the  FedEx  case, 
discussions  regarding  important  business  decisions  are  expected  to  be 
supported  by  corporate  or  line-of-business  specific  OR  analyses. 
Recommendations  not  supported  risk  being  dismissed. 

•  In  both  the  Army  and  FedEx,  OR  is  regarded  as  a  discreet  career  field.  OR 
professionals  are  recruited,  and  in  the  case  of  the  Army,  developed.  OR 
professionals  in  both  organizations  enjoy  respect;  career  prospects;  and  the 
opportunity  to  develop,  refine,  and  demonstrate  their  capabilities.  In  the 
other  Services,  the  Task  Force  found  that  a  career  in  Operations  Research  can 
lead  to  careers  removed  from  the  mainstream  resulting  in  diminished  career 
prospects. 

•  The  Army  and  FedEx  use  OR  broadly.  The  TRAC  mission  unites  investment 
decision  making  with  operational  concepts.  While  many  companies  employ 
OR  in  decision  making,  the  Task  Force  found  the  FedEx  model  very  relevant 
to  the  functioning  of  DoD.  FedEx  uses  OR  to  help  decide  in  which  businesses 
to  compete  and  how  to  allocate  resources  within  specific  lines  of  business  in 
support  of  current  operations.  In  the  FedEx  process  model,  OR  proves  its 
value  constantly. 

•  The  Army  and  FedEx  appear  to  have  aligned  closely  their  respective  OR 
cadres,  decision  makers,  and  operators.  Analyses  employing  OR  appear  to 
reflect  current  operational  experience.  The  Task  Force  notes  that  this  factor 
is  also  demonstrated  in  the  historical  context  in  which  OR  has  been  used. 

In  the  past  DoD  tried  to  institutionalize  structured  ISR  decision  support.  In  the  mid- 
1990's  it  established  a  Joint  C4ISR  Decision  Support  Center  (the  Center).  In  particular, 
the  FY97  Defense  Planning  Guidance  (DPG)  directed  that  the  Office  of  the  Assistant 
Secretary  of  Defense  for  Command,  Control,  Communications  and  Intelligence  (OASD 
(C3I))  lead  an  effort  in  coordination  with  the  USD(AT&L),  Joint  Staff,  CINCS,  Services  and 
Agencies  to  develop  a  DoD  Joint  C4ISR  Conceptual  Architecture  which  would  identify 
opportunities  to  invest  in  Joint  and  integrated  C4ISR  Systems.  The  purpose  of  this 
directive  was  to  increase  efficiency  in  the  performance  of  C4ISR  systems  in  Joint 
operations  and  to  reduce  the  apparent  duplicative  expenditures  in  C4ISR  systems  across 
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the  Department  of  Defense.  $10M  per  year  was  made  available  for  FY97-2001  in 
support  of  this  effort.  The  Center's  challenge  was  specific;  it  was  expected  to  provide 
analysis  results  to  the  Joint  C4ISR  DSC  Senior  Steering  Group,  Joint  Requirements 
Oversight  Council  (JROC)  and  the  Defense  Acquisition  Board  (DAB)  to  assist  senior 
decision  makers  in  making  Joint  C4ISR  investment  decisions. 

The  Center  was  used  actively  in  support  of  a  number  of  studies  including  Sensor-to- 
Shooter,  Precision  Force  C4I  Operational  Architecture,  and  other  efforts.  The  Center's 
efforts,  however,  were  not  generally  coupled  to  acquisition  strategies.  Over  time,  the 
Center's  resources  diminished  and,  while  it  still  exists,  the  Center  conducts  fewer 
studies  and  enjoys  limited  visibility.  Overall,  the  Center  lacked  sustained  leadership 
support,  resources,  and  the  mandate  to  conduct  meaningful  analyses  coupled  to 
acquisition  and  investment  decisions. 

The  Task  Force  recognizes  that  several  organizational  models  are  available  to  the  USD(I) 
to  institutionalize  Operations  Research  (and  scientific  method-based  decision-support) 
that  can  address  these  problems.  The  Task  Force  urges  the  USD(I)  and  the  Department 
to  look  closely  at  the  private  sector  and  the  FedEx  model,  one  in  which  there  exists 
"corporate"  governance  regarding  the  necessity  to  employ  OR  for  key  decisions, 
enforcement  of  OR  standards  processes  and  practices,  oversight  of  line-of-business- 
specific  OR  capabilities,  and  evaluation  of  the  quality  of  OR  products.  The  company's 
approach  encompasses  a  compact,  but  influential  corporate  organization  (for 
governance,  oversight,  and  OR  analysis  in  support  of  corporate  strategy  decisions),  and 
more  fully  elaborated  OR  organizations  embedded  in  the  lines  of  business.  The  Army's 
TRAC  organization  may  represent  a  sound  example  of  such  an  embedded  organization, 
performing  a  wide  range  of  decision-support  analyses.  Flowever  the  USD(I)  proceeds, 
the  MORS  represents  a  powerful  domain  expert  resource  that  can  help  build  a  plan  to 
institutionalize  OR. 
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Chapter  8:  Findings  and  Recommendations 

The  USD(I)  should  recognize  explicitly  that  the  value,  quality,  and  use  of  OR  in  support  of 
a  given  decision  is  dependent  on  the  decision  maker's  perception  of  its  value  and 
relevance;  that  the  quality  of  ISR  decisions  can  be  improved  by  increased  use  of  high 
quality  OR  in  decision  making;  and  that  the  state  of  OR  is  advancing. 

OR  often  gets  attention  on  an  ad  hoc  basis,  but  when  immediate  demand  subsides, 
interest  in  sustaining  its  use  tends  to  wane.  Institutionalization  of  OR  is  missing  in  the 
overall  decision-making  process.  It  is  important  to  develop  a  set  of  required  standard 
processes  and  practices,  milestones,  methodologies,  and  thresholds  for  objective 
application  of  OR  to  high-value  decisions.  The  Task  Force  recommends  the  USD(I) 
closely  examine  the  FedEx  and  TRAC  models.  Buttressing  these  views  are  four 
important  observations: 

1.  Operations  Research  represents  a  powerful  tool  to  help  improve  the  quality 
of  investment  decision  making  by  illuminating  key  issues,  assumptions,  and 
sources  of  information. 

2.  Operations  Research  is  applied  inconsistently  throughout  the  Defense  and 
ISR  communities.  These  communities  do  not  possess  standard  processes  and 
practices  relating  to  OR  practice,  a  consistent  organizational  model,  or  a 
consistent  commitment  to  the  use  of  OR. 

3.  OR  -  and  its  use  -  can  be  strengthened  in  the  Defense  and  ISR  communities 
through  effective  institutionalization. 

4.  The  utility  of  OR  can  be  more  firmly  established  through  appropriate  test 
cases. 

The  USD(I)  should  employ  OR  to  support  ISR  investment  decisions  relating  to  the 
dynamic  set  of  capabilities  found  throughout  the  intelligence  cycle  that  need  to  be 
maintained,  upgraded,  developed,  or  acquired  to  satisfy  the  changing  requirements  that 
the  intelligence  cycle  must  support.  This  OR  capability  should  be  organic  to  the  Office  of 
the  USD(I),  enjoy  the  full  confidence  and  support  of  the  USD(I),  and  be  comprised  of  OR 
professionals  who  can  provide  objective  OR-based  recommendations  to  guide  ISR 
investment  decisions. 

The  Task  Force  offers  the  following  recommendations: 

1.  As  Battlespace  Awareness  portfolio  manager,  USD(I)  should: 

1.1.  Establish  an  organic  ISR  OR  oversight  function  that  requires,  sustains,  and 
reviews  OR  standard  processes  and  practices  in  DoD  ISR  investment  decisions. 

•  Consider  the  approach  employed  by  FedEx  as  an  institutional  OR  model, 
i.e.,  decentralized  OR-based  decision  support  performed  in  lines  of 
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business;  a  compact  corporate  OR  organization  provides  standard 
processes  and  practices,  and  oversight  of  OR  use,  and  conducts  OR  on 
strategic  corporate  issues. 

•  Consider  MORS  as  a  domain  expert  resource  to  assist  in  ISR  OR 
institutionalization. 

1.2.  Establish  a  USD(I)  capability  to  conduct  OR  for  strategic  investment  decisions 
(includes  delegation  of  specific  OR  analyses  to  subordinate  ISR  organizations). 

•  Create  a  professional  OR  community  of  interest  and  practice  across  DoD 
and  the  1C. 

•  Coordinate  development  and  application  of  OR  models  for  specific  ISR 
investment  decisions. 

1.3.  Define  requirements  for  broad  based  studies  for  decision  making  milestones 
and  a  joint  capability  for  accrediting  the  studies,  models,  and  data  used  and 
develop  best  practices/models  for  ISR  effects. 

1.4.  Ensure  OR  requirements  for  MAPs  and  programs  below  Milestone  Decision 
Authority  are  understood  and  met. 

2.  This  OUSD(I)  OR  function  should: 

2.1.  Serve  as  the  implementation  mechanism  for  the  institutionalization  of  ISR  OR 
within  the  DoD. 

2.2.  Advocate  OR  as  a  required  input  to  ISR  decision-making  processes. 

2.3.  Oversee  the  application  of  OR  to  run  test  cases  and  scenarios. 

2.4.  Apply  OR  to  validate  assumptions. 

2.5.  Use  OR  to  test  investment  decisions  for  proper  balance. 

2.6.  Track  the  state  of  OR  and  improve  OR  tools  supporting  ISR  decisions  and 
employ  them  in  the  decision-making  process. 

3.  Test  Cases: 

3.1.  In  response  to  the  TOR,  the  Task  Force  recommends  two  test  cases:  one 
related  to  investment  decision  making  in  biometrics  technologies,  and  one  related 
to  the  balance  in  the  intelligence  cycle.  These  test  cases  should  be  structured  to 
illuminate  challenges  and  opportunities  to  strengthen  OR  decision  making  in  the  ISR 
investment  domain. 
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ACQUISITION, 
TECHNOLOGY 
AND  LOGISTICS 


THE  UNDER  SECRETARY  OF  DEFENSE 

3010  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3010 


JUL  0  5  2007 


MEMORANDUM  FOR  CHAIRMAN,  DEFENSE  SCIENCE  BOARD 


SUBJECT:  Terms  of  Reference  -  Defense  Science  Board  (DSB)  Advisory  Group  on  Defense 
Intelligence 


You  are  directed  to  form  a  permanent  DSB  Advisory  Group  to  provide  study  support 
and  independent  advice  to  the  Under  Secretary  of  Defense  for  Intelligence  on  a  broad  range 
of  defense  intelligence  matters.  The  initial  review  area  will  be  on  Improving  Intelligence 
Systems  Acquisitions.  Additional  study  areas  will  be  formulated  over  time  to  support  the 
evolving  needs  of  the  defense  intelligence  enterprise. 

The  Advisory  Group  will  report  via  the  full  DSB  to  the  interested  Federal  Official. 
Acquisition  related  matters  will  be  reported  to  both  USD(AT&L)  and  USD(I),  and 
intelligence  related  matters  to  USD(I). 

At  least  one  member  of  the  DSB  must  be  a  member  of  the  Advisory  Group  and  any 
sub  groups  formed  under  the  Advisory  Group. 

The  Task  Force  shall  have  access  to  all  classified  information,  including  special 
access  programs,  relevant  to  the  performance  of  its  tasks. 

Major  General  Richard  O’Lear,  USAF  (ret.)  will  serve  as  the  Chairman  of  this  Task 
Force.  Ms.  Betty  Sapp,  OUSD(I),  will  function  as  the  primary  Designated  Federal  Official. 
USD(I)  will  coordinate  on  all  proposed  Task  Force  members. 

The  Task  Force  will  operate  in  accordance  with  the  provisions  of  Public  Law  92-463, 
the  “Federal  Advisory  Committee  Act”  and  DoD  Directive  5105.4,  the  “DoD  Federal 
Advisory  Committee  Management  Program”.  We  do  not  anticipate  that  this  Task  Force  will 
need  to  go  into  any  ‘particular  matters’  within  the  meaning  of  Title  18,  United  States  Code, 
section  208,  nor  will  it  cause  any  member  to  be  placed  in  the  position  as  a  procurement 
official. 

This  Task  Force  will  be  renewed  annually  or  upon  change  of  the  Task  Force 
Chairman  or  Designated  Federal  Official. 


Kenneth  J.  Krieg 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3000 

JAN  2  9  20no 

INTELLIGENCE 

MEMORANDUM  FOR  CHAIRMAN.  DEFENSE  SCIENCE  BOARD  ADVISORY 

GROUP  ON  DEFENSE  INTELLIGENCE 

SUBJECT:  Terms  of  Reference  -Operations  Research  Applications  for  Intelligence, 
Surveillance  and  Reconnaissance  (ISR) 

You  are  requested  to  perform  a  DSB  Advisory  Group  on  Defense  Intelligence 
study  to  examine  the  use  of  operations  research  (OR)  methods  to  support  decision¬ 
making  within  the  Department,  with  a  focus  on  ISR  investment  choices. 

The  principal  objective  is  to  assess  the  use  of  OR  methods  within  the  Department, 
determine  how  OR  supports  Department  decision-making  for  ISR  programs  and  to 
recommend  improvements,  if  needed. 

DoD  ISR  is  a  large,  complex  enterprise  managed  by  the  Under  Secretary  of 
Defense  for  Intelligence  (USD(I)).  While  primary  stakeholders  for  DoD  ISR  are  the 
Combatant  Commands,  global  ISR  requirements  come  from  many  sources  and  the  low 
volume,  high  demand  DoD  ISR  assets  arc  continually  budget-constrained.  Recent  use  of 
OR  analysis  to  inform  the  High  Value  Target  study  has  received  positive  feedback,  and 
there  is  a  wealth  of  operational  data  that  could  be  used  to  expand  this  analysis  across  the 
ISR  spectrum  of  requirements. 

The  “OR  Methods  ISR  Study"  should  proceed  in  four  stages: 

( 1 )  Survey  the  departments  and  agencies  to  determine  how  much  OR  is  being 
performed; 

(2)  Assess  how  the  results  of  that  OR  are  being  used  in  decision  making; 

(3)  Based  on  the  findings  of  “( I )”  and  "(2)",  recommend  a  test  case  for  using  OR 
methodologies  to  improve  DoD  ISR  investment  decisions; 

(4)  Recommend  steps  the  Department  can  take  to  "institutionalize"  the  use  of  OR 
in  DoD  decision  making  in  the  future. 

The  study  will  be  sponsored  by  the  Under  Secretary  of  Defense  for  Intelligence. 
USD(I)  is  authorized  to  act  upon  the  advice  and  recommendations  of  the  Board. 

CAPT  Keith  May,  OUSD(I),  will  serve  as  the  Executive  Secretary. 
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The  Task  force  will  operate  in  accordance  with  the  provisions  of  1\L.  92-463,  the 
“Federal  Advisor)'  Committee  Act,”  and  DoD  Directive  5105.4,  the  “DoD  Federal 
Advisory  Committee  Management  program."  It  is  not  anticipated  that  this  Task  Force 
will  need  to  go  into  any  “particular  matters”  within  the  meaning  of  Section  208  of  Title 
18,  United  States  Code,  nor  will  it  cause  any  member  to  be  placed  in  the  position  of 
action  as  a  procurement  official. 


Betty  J.  Sapp 

Deputy  Under  Secretary'  of  Defense 
(Acquisition,  Resources  &  Technology) 
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D.l  Biometrics 

The  Secretary  of  Defense  (SecDef)  has  recently  taken  steps  to  organize  DoD  biometrics 
activities.  Department  of  Defense  Directive  (DoDD)  8521. 01E  of  February  21,  2008 
"...designates  the  Director,  Defense  Research  &  Engineering  (DDR&E)  as  the  Principal 
Staff  Assistant  (PSA)  responsible  for  oversight  of  DoD  biometrics  programs  and  policy..." 
8521. 01E  also  "...designates  the  Secretary  of  the  Army  as  the  DoD  Executive  Agent  (EA) 
for  DoD  biometrics..."  Section  5.3  of  the  Directive  assigns  to  the  Under  Secretary  of 
Defense  for  Intelligence  (USD(I))  a  broad  range  of  responsibilities  to  ensure  biometrics 
are  used  effectively  to  meet  intelligence  and  counter-intelligence  challenges.34 

Although  the  Directive  clarifies  DoD  Biometrics  responsibilities,  questions  remain 
regarding  the  application  of  biometrics  to  military  operations  in  general,  and  to  the  ISR 
domain  in  particular.  Although  the  Administration  has  requested  funding  from  Congress 
for  biometrics  applications  and  the  Army  has  established  a  Biometrics  Fusion  Center 
(BFC),  no  clear  path  exists  yet  for  investment  in  biometrics.  Concepts  of  operations 
(CONOPS)  for  application  of  biometrics  attributes  are  lacking,  as  are  the  requirements 
important  to  making  biometrics  investment  decisions.  As  a  result,  the  DSB  Advisory 
Group  on  Defense  Intelligence  regards  biometrics  as  an  appropriate  test  case  for  the  use 
of  Operations  Research  in  ISR  investment  decision  making. 

In  the  Task  Force's  view,  OR  can  be  used  in  three  ways  pertinent  to  biometrics 
investment  decision  making,  each  of  which  would  help  the  Department  understand  OR's 
potential  as  a  decision-support  discipline  and  lead  to  concrete  results  in  the  short  term 
relevant  to  biometrics. 

First,  the  Department  can  employ  OR  to  establish  and  refine  concepts  of  operations  for 
the  application  of  biometrics,  end-to-end,  in  support  of  military  operations  in  general 
and  to  meet  ISR  requirements  in  particular.  OR  has  been  used  traditionally  by  the 
Services  to  model  operational  requirements,  desired  effects,  and  the  application  of 
resources  to  meet  those  requirements  and  achieve  those  effects.  OR  can  be  used  to 
help  refine  the  understanding  of  the  ways  in  which  the  insertion  of  specific  biometrics 
data  can  support  and  improve  the  effectiveness  of  specific  military  operations.  In  this 
regard,  OR  can  also  help  improve  the  architecture  of  military  operations  that  employ 
biometrics. 

Second,  the  USD(I)  can  work  with  the  Office  of  the  Director  of  National  Intelligence 
(ODNI)  to  strengthen  the  information  architecture  associated  with  biometrics 
databases,  information  sharing,  as  well  as  biometrics  collection,  analysis,  and 
dissemination.  While  laudable  work  has  already  been  done  at  the  National 
Counterterrorism  Center  (NCTC),  Federal  Bureau  of  Investigation  (FBI),  and  elsewhere  in 


34  Department  of  Defense  Directive  (DoDD)  8521.01E,  dated  February  21,  2008. 
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the  Intelligence  Community  to  employ  biometrics  information,  a  unified  information 
architecture  for  the  application  of  biometric  attributes  (i.e.,  biometrics)  by  the 
Intelligence  Community  does  not  exist.  Work  remains  to  be  done  to  build  capabilities  to 
query  a  wide  variety  of  biometrics  databases  on  a  federal  basis.  Indeed,  information 
sharing  schemes  remain  rudimentary,  dependent  on  security  restrictions,  and  hampered 
by  a  lack  of  data  standards.  Nonetheless,  investment  continues  in  a  wide  variety  of 
programs  associated  with  counter-terrorism  and  war-fighter  support,  many  of  them 
dependent  on  the  availability  and  use  of  biometrics  data.  In  addition,  the  1C  needs,  but 
does  not  possess,  a  sound  analytical  concept  for  the  use  of  biometric  attributes,  a  fact 
that  can  also  impair  the  ability  of  the  1C  to  define  biometrics  ISR  requirements  and  an 
effective  information  architecture  that  encompasses  the  use  of  biometrics.  Indeed,  the 
Task  Force  is  aware  of  broad  concerns  that  biometrics  collection  capabilities  have 
outpaced  the  ability  of  the  Intelligence  and  Defense  Communities  to  share  and  employ 
biometrics  data,  both  for  analysis  and  for  tactical  military  operations.  As  a  result,  the 
"promise"  of  biometrics  has  yet  to  be  fulfilled,  even  through  biometrics  collection 
capabilities  (e.g.,  retinal,  facial,  and  voice  recognition)  and  tools  to  "enroll"  and  compare 
an  individual  with  biometrics  data  (e.g.,  the  Biometrics  Automated  Toolset)  have 
continued  to  develop. 

OR  should  be  assessed  as  a  mechanism  for  defining  intelligence  operations  including 
tasking,  collection,  and  analysis  pertinent  to  biometrics  data,  and  dissemination  of 
intelligence  products  encompassing  a  biometrics  component.  Such  an  application  of  OR 
could  have  several  distinct  benefits: 

•  It  can  lead  to  better.  Community-wide  biometrics  information  architecture. 

•  It  can  help  the  Community  achieve  true  biometrics  information  sharing. 

•  It  can  help  the  USD(I)  and  others  understand  how  OR  might  be  employed  to 
build  information-centric  inter-agency  and  Community-wide  ISR  capabilities. 

Third,  the  Secretary  of  the  Army  and  the  BFC  can  use  OR  to  conduct  trade  studies  on  a 
variety  of  specific  biometrics  technologies  and  acquisition  strategies,  consistent  with 
DoD  acquisition  program  management  requirements.  OR  can  be  used  to  compare 
Technology  Readiness  Levels,  expected  investment  levels,  and  likely  operational 
capabilities  associated  with  competing  technologies,  acquisition  and  investment 
strategies,  and  funding  arrangements.  In  this  instance,  OR  would  employ  biometrics  as 
a  technology  development  and  acquisition  strategy  test-bed.  Such  an  approach  is 
particularly  appropriate,  given  recent  moves  to  strengthen  DoD  biometrics  oversight, 
enhance  the  financial  resources  associated  with  biometrics  investment,  and  upcoming 
acquisition  activities  on  the  part  of  the  Biometrics  Fusion  Center.  In  this  instance,  the 
Department  should  choose  an  upcoming  procurement  activity  related  to  biometrics 
collection  or  enrollment  and  employ  OR  to  help  define  and  frame  a  set  of  investment 
decisions  related  to  two  or  more  specific  technologies  and  two  or  more  specific 
acquisition  strategies. 
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Overall,  the  three  elements  described  above  could  help  the  USD(I)  understand  how  OR 
could  improve  warfighter  operations  through  the  use  of  biometrics  as  a  component  of 
ISR,  lead  to  a  stronger  analytic  concept  for  the  use  of  biometrics,  contribute  to  the 
development  of  a  biometric  information  architecture,  and  strengthen  biometrics 
technology  and  program  investment  decision  making. 

D.2  Investing  in  a  Balanced  Intelligence  Cycle  and  Imagery  UASs  -  a  Value  Proposition 

The  Intelligence  Community  has  a  history  of  investing  in  new,  discipline-specific 
capabilities  without  proper  consideration  of  the  impact  of  decisions  on  the  entire 
intelligence  cycle.  In  the  past  this  resulted  in  large  outlays  of  resources  that,  because  of 
this  imbalance,  were  either  not  very  useful  or  were  marginalized  because  of  an  inability 
to  realize  full  value  from  the  investment.  For  purposes  of  this  discussion,  the  Task  Force 
defines  the  "Intelligence  Cycle"  as  follows:  Planning  and  Direction,  Collection, 
Exploitation,  Analysis,  Dissemination,  and  Feedback  (Note:  The  Task  Force  recognizes 
that,  depending  upon  the  collection  system  being  considered,  dissemination  may  occur 
at  multiple  places  in  this  cycle,  for  example  between  collection  and  exploitation,  again 
between  exploitation  and  all-source  analysis,  and  finally  to  the  end-user).  Some  historic 
examples  of  where  cycle  imbalance  caused  significant  problems  or  challenges  include 
the  following: 

1.  Inability  to  adequately  disseminate  national  imagery  to  tactical  units  during 
Desert  Shield/Desert  Storm.  This  is  perhaps  the  most  notable  example  of  where 
a  lack  of  balance  in  the  intelligence  cycle  made  a  magnificent  collection  product 
not  available  in  a  timely  fashion  to  warfighters  in  the  volume  or  time-frame 
needed.  In  this  case  an  enormous  investment  in  imagery  collection  capability 
was  not  matched  by  the  investment  needed  in  dissemination  capability  or 
technology.  The  result  was  air  strike  target  imagery  or  amphibious  landing  zone 
imagery  was  available  at  national  and/or  combatant  command  levels,  but  not  to 
those  with  the  mission  of  striking  the  targets  or  landing  on  the  beaches.  In 
addition,  even  if  dissemination  had  been  much  more  capable,  imagery  analysts 
were  not  available  in  the  numbers  needed  for  exploitation  of  the  imagery 
product.  The  reasons  for  this  imbalance  are  multiple  (failure  to  consider 
dissemination  or  analysis  requirements  by  a  collection  system  acquirer,  failure  to 
properly  invest  in  dissemination  capability  or  technology  by  military  Service 
users)  so  the  purpose  here  is  not  to  try  to  establish  blame,  but  to  point  out  that 
an  OR  approach  to  examining  the  dissemination  and  exploitation  issues  at  the 
time  of  collection  system  investment  decision  would  have  been  helpful. 

2.  Inability  to  adequately  provide  timely  feedback  to  collection  system  planning 
and  direction,  especially  when  the  feedback  to  planning  and  direction  reaches 
across  collection  disciplines.  This  challenge  is  often  referred  to  as  "cross- 
cueing,"  and  it  is  one  that  is  rarely  provided  for  in  the  design  and  procurement  of 
a  collection  system.  U.S.  war-fighting  history  is  replete  with  examples  of  where 
adequate  and  timely  feedback  from  one  collection  discipline  to  another  (notably 
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among  and  between  SIGINT,  ELINT,  and  IMINT)  might  have  resulted  in  high-value 
target  location  or  important  intelligence  discovery. 

3.  Inability  to  adequately  provide  specialized  analysis  to  support  new  collection 
capabilities.  Measurement  and  Signature  Intelligence  (MASINT)  is  an  area  where 
new  technical  collection  capability  and  capacity  may  be  significantly  outpacing 
the  ability  to  realize  the  full  value  of  new  kinds  of  technical  collection.  Many 
MASINT  products  require  specialized  exploitation  and  specifically  trained  and 
experienced  analysts  to  make  good  use  of  products.  It  is  a  problem  for 
intelligence  cycle  effectiveness  and  utility  if  investment  in  exploitation  and 
analysis  for  MASINT  does  not  match  the  developing  collection  capacity  and  it  is 
probably  better  to  re-balance  the  investment  than  continue  to  pour  more 
MASINT  data  into  the  system.  An  historical  example  of  where  an  effective 
MASINT  intelligence  cycle  was  achieved  through  balanced  investment  in 
planning  and  direction,  collection,  exploitation,  analysis,  dissemination  and 
feedback  was  the  U.S.  Navy's  use  of  Acoustic  Intelligence  (ACINT)  during  the  Cold 
War.  Over  several  decades,  very  large  investment  by  the  Navy  -  in  both 
operations  and  intelligence  -  resulted  in  strong  technical  and  tactical  ACINT 
collection  matched  by  exploitation  of  the  data  and  use  in  all-source  intelligence 
analysis  to  provide  cueing  information  to  operational  forces  and  technical 
information  to  weapons  systems  developers.  Coincidentally,  this  same  highly 
developed  (and  expensive)  system  (SOSUS,  ASWOCS,  FOSICS,  and  FOSIFS)  made 
very  good  use  of  cross-cueing  techniques  between  ACINT  and 
IM I NT/SIGI NT/ELI  NT.  U.S.  Navy  anti-submarine  warfare  is  strongly  rooted  in  a 
history  of  Operations  Research,  and  the  degree  to  which  OR  played  a  part  in 
developing  the  ASW/ACINT  system  might  be  a  useful  case  study  in  its  own  right. 

Significant  discussion  is  underway  in  the  Intelligence  Community  regarding  the 
appropriate  balance  of  resource  investment  in  the  Tasking,  Processing,  Exploitation,  and 
Dissemination  (TPED)  segments  of  ISR  systems.  The  Task  Force  suggests  that  to  look 
only  at  TPED  may  be  "shooting  short  of  the  mark."  Instead,  a  test  case  using  OR 
techniques,  tools,  and  practitioners  analyze  an  ISR  resource  decision  with  the  goal  of 
optimizing  allocation  of  resources  in  a  balanced  way  across  the  entire  intelligence  cycle 
as  described  in  paragraph  1  above.  A  potential  test  case  would  be  to  look  at  a  planned 
investment  in  imagery  UASs,  whose  collection  potential  can  surely  be  examined  in 
detail  using  valid  coverage  factors  and  collection  capability.  However,  all  the 
additional  parts  of  the  intelligence  cycle  should  be  analyzed  similarly  with  a  view 
toward  answering  the  following  questions  about  other  parts  of  the  intelligence  cycle 
relevant  to  the  potential  UAS  collection  system  investment. 

1.  Planning  and  direction:  What  is  the  command  and  control  apparatus  and  are 
there  the  right  number  of  sufficiently  trained  personnel  in  that  system  to  do 
mission  planning  relevant  to  the  type  of  missions  envisioned?  If  not,  how  many 
are  needed  and  what  is  the  cost  associated  with  recruiting,  training,  and 
retaining  them?  Are  the  planned  UASs  fully  compatible  with  existing  ground 
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station  infrastructure  and  is  there  sufficient  capacity  in  them  to  handle  the 
envisioned  new  system?  If  ground  station  additions/modifications  and 
additional  ground  station  personnel  are  required,  have  the  costs  been  fully 
analyzed  and  included  in  the  planned  investment? 

2.  Collection:  Even  assuming  that  the  collection  capability  of  the  planned  UAS 
system  has  been  correctly  measured  and  modeled,  have  the  costs  of  future 
potential  upgrades  and/or  alternative  collection  capabilities  been  estimated? 

3.  Dissemination:  Is  the  contemplated  new  UAS  collection  capability  fully 
compatible  with  existing  dissemination  networks  and  is  there  sufficient 
communications  capacity  to  handle  the  increased  volume  of  collection? 
Additionally,  once  exploitation  has  been  conducted,  is  there  sufficient  secondary 
dissemination  capacity  to  get  exploited  data  into  the  hands  of  users?  The 
bottom  line  is  to  ensure  that  there  is  sufficient,  credible  analysis  to  answer  the 
questions  relative  to  primary  and  secondary  dissemination  requirements,  and 
that  associated  cost  data  are  included  in  the  investment  planning. 

4.  Exploitation:  An  historic  problem  associated  with  imagery  collection  systems  is  a 
failure  to  plan  and  pay  for  sufficient  imagery  exploitation  personnel  to  handle 
the  increased  data  quantity.  An  OR  analysis  approach  could  help  answer  this 
question,  as  well  as  provide  cost  estimates  of  what  the  additional  personnel  and 
their  training  would  cost  so  that  these  costs  could  be  included  in  the  cost  of  the 
overall  procurement  plan. 

5.  Analysis:  There  is  a  difference  between  exploitation  and  analysis,  and  if  the 
planned  new  collection  capability  is  to  fully  contribute  to  actual  knowledge 
development,  the  capacity  must  exist  to  disseminate  and  incorporate  the 
exploited  information  into  the  all-source  analysis  system.  While  all-source 
analysis  is  not  a  part  of  the  collection  system  per  se,  pre-decisional  OR  analysis 
should  be  conducted  to  ensure  that  the  envisioned  system  is  fully  compatible 
with  all  source  analysis  requirements  in  all  parts  of  the  ISR  system  that  could 
benefit  from  the  information. 

6.  Feedback:  Often  the  most  overlooked  part  of  the  intelligence  cycle,  any  new 
system  acquisition  should  include  plans  for  providing  feedback  -  to  the  entire  ISR 
system  if  required.  Perhaps  the  most  sophisticated  form  of  feedback  is  "cross- 
cueing,"  where  exploited  data  from  one  collection  system  can  immediately  cue 
(or  be  cued  by)  other  collection  systems.  This  type  of  feedback  is  critical  to  an 
effective  and  efficient  ISR  system  and  has  rarely  been  incorporated  in  system 
acquisition  decisions,  but  should  be  analyzed  and  accounted  for  during 
investment  decision  deliberations. 
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ACINT 

Acoustic  Intelligence 

AFIT 

Air  Force  Institute  of  Technology 

AOR 

Area  of  Responsibility 

ARCIC 

Army  Capabilities  Integration  Center 

A/S 

Assistant  Secretary 

ASD 

Assistant  Secretary  of  Defense 

ASW 

Anti-Submarine  Warfare 

ASWORG 

Anti-Submarine  Warfare  Operations  Research  Group 

BA 

Battlespace  Awareness 

BAT 

Biometrics  Automated  Toolkit 

BDC 

Biometrics  Fusion  Center 

BFC 

Biometrics  Fusion  Center 

C3I 

Command,  Control,  Communications,  and  Intelligence 

C4ISR 

Command,  Control,  Communications,  Computers,  Intelligence, 
Surveillance  and  Reconnaissance 

CAA 

Center  for  Army  Analysis 

CIA 

Central  Intelligence  Agency 

CINCs 

Commanders  in  Chief 

CJCS 

Chairman  of  the  Joint  Chiefs  of  Staff 

COCOM 

Combatant  Command 

CONOPS 

Concept  of  Operations 

DAB 

Defense  Acquisition  Board 

DARPA 

Defense  Advanced  Research  Projects  Agency 

DDRE 

Director  of  Defense  Research  and  Engineering 

DEPSECDEF 

Deputy  Secretary  of  Defense 

DFAR 

Defense  Federal  Acquisition  Regulation 

DIA 

Defense  Intelligence  Agency 

DIOCC 

Defense  Intelligence  Operations  Coordination  Center 

DNI 

Director  of  National  Intelligence 
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DoD 

Department  of  Defense 

DoDD 

DoD  Directive 

DPG 

Defense  Planning  Guidance 

DSB 

Defense  Science  Board 

DTED 

Digital  Terrain  Elevation  Data 

DTRA 

Defense  Threat  Reduction  Agency 

EA 

Executive  Agent 

ELINT 

Electromagnetic  Intelligence 

FAR 

Federal  Acquisition  Regulations 

FBI 

Federal  Bureau  of  Investigation 

FedEx 

Federal  Express 

FFRDC 

Federally  Funded  Research  and  Development  Center 

FIA 

Future  Imagery  Architecture 

FMV 

Full  Motion  Video 

FYDP 

Future  Year  Defense  Plan 

GWOT 

Global  War  on  Terror 

HUMINT 

Fluman  Intelligence 

HVT 

High  Value  Target 

1C 

Intelligence  Community 

ICDO 

Integrated  Concepts  Development  Office 

IOSA 

Integrated  Overhead  SIGINT  Architecture 

ISR 

Intelligence,  Surveillance,  and  Reconnaissance 

JASTP 

Joint  Advanced  Strike  Technology  Program 

JCS 

Joint  Chiefs  of  Staff 

JDAM 

Joint  Direct  Attack  Munition 

JROC 

Joint  Requirements  Oversight  Council 

JSF 

Joint  Strike  Fighter 

JWCA 

Joint  Warfighting  Capability  Assessment 

MAP 

Major  Acquisition  Program 

MASINT 

Measurement  and  Signature  Intelligence 

MIT 

Massachusetts  Institute  of  Technology 
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MOA 

Memorandum  of  Agreement 

MORS 

Military  Operations  Research  Society 

MOU 

Memorandum  of  Understanding 

MWORG 

Mine  Warfare  Operations  Research  Group 

NCTC 

National  Counterterrorism  Center 

NDRC 

National  Defense  Research  Committee 

NGA 

National  Geospatial-Intelligence  Agency 

NIC-C 

National  Intelligence  Coordination  Center 

NPS 

Naval  Postgraduate  School 

NSA 

National  Security  Agency 

NRO 

National  Reconnaissance  Office 

OASD 

Office  of  the  Assistant  Secretary  of  Defense 

ODNI 

Office  of  the  Director  of  National  Intelligence 

OIF 

Operation  Iraqi  Freedom 

OR 

Operations  Research 

ORS 

Operational  Research  Section 

OSD 

Office  of  the  Secretary  of  Defense 

OSRD 

Office  of  Scientific  Research  and  Development 

OUSD(AT&L) 

Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics 

OUSD(I) 

Office  of  the  Under  Secretary  of  Defense  for  Intelligence 

PA&E 

Program  Analysis  and  Evaluation 

PPBS 

Planning,  Programming  and  Budgeting  System 

PSA 

Principal  Staff  Assistant 

R&D 

Research  and  Development 

RAF 

Royal  Air  Force 

RDT&E 

Research,  Development,  Test  and  Evaluation 

S&T 

Science  and  Technology 

SA 

Systems  Analysis 

SDD 

System  Design  and  Development 

SecDef 

Secretary  of  Defense 

Operations  Research  Applications  for  ISR 
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SIGINT 

Signals  Intelligence 

SIOP 

Single  Integrated  Operational  Plan 

SOF 

Special  Operations  Forces 

SOSUS 

Sound  Surveillance  System 

TOR 

Terms  of  Reference 

TPED 

Tasking,  Processing,  Exploitation,  Dissemination 

TRAC 

TRADOC  Analyses  Center 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

TSWG 

Technical  Support  Working  Group 

UAS 

Unmanned  Aerial  System 

USA 

United  States  Army 

USAF 

United  States  Air  Force 

USD(AT&L) 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 

USD(I) 

Under  Secretary  of  Defense  for  Intelligence 

USMC 

United  States  Marine  Corps 

USN 

United  States  Navy 

U.S. 

United  States 

VCJCS 

Vice  Chairman  of  the  Joint  Chiefs  of  Staff 
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